1
00:00:13,430 --> 00:00:11,270
hey good evening ladies and gentlemen

2
00:00:14,910 --> 00:00:13,440
how's everyone tonight

3
00:00:17,269 --> 00:00:14,920
good thanks for coming out to join us

4
00:00:19,429 --> 00:00:17,279
tonight what would it be like to swim in

5
00:00:22,310 --> 00:00:19,439
the oceans of europa and what would it

6
00:00:23,750 --> 00:00:22,320
feel like to climb olympus mons on mars

7
00:00:25,910 --> 00:00:23,760
and would it be possible for all of us

8
00:00:28,150 --> 00:00:25,920
to experience these journeys together

9
00:00:30,230 --> 00:00:28,160
the goal of the tele exploration program

10
00:00:31,830 --> 00:00:30,240
is to make us better explorers by

11
00:00:33,910 --> 00:00:31,840
building immersive environments that

12
00:00:35,670 --> 00:00:33,920
feel like we're really there

13
00:00:37,670 --> 00:00:35,680
by drawing together technologies from

14
00:00:38,869 --> 00:00:37,680
sources as unlikely as the video game

15
00:00:40,150 --> 00:00:38,879
industry

16
00:00:41,830 --> 00:00:40,160
and

17
00:00:44,229 --> 00:00:41,840
advancing the state of the art pardon me

18
00:00:46,630 --> 00:00:44,239
in a human system interaction we are

19
00:00:48,709 --> 00:00:46,640
working towards low-cost holodecks not

20
00:00:50,310 --> 00:00:48,719
only for every nasa mission scientist

21
00:00:52,470 --> 00:00:50,320
but for every person who longs to

22
00:00:53,990 --> 00:00:52,480
explore space with us

23
00:00:55,750 --> 00:00:54,000
this presentation will describe our

24
00:00:58,470 --> 00:00:55,760
progress towards these ambitious goals

25
00:01:00,389 --> 00:00:58,480
as well as the challenges that are ahead

26
00:01:02,470 --> 00:01:00,399
tonight's guest received his bachelor's

27
00:01:05,109 --> 00:01:02,480
and master's degrees in computer science

28
00:01:06,149 --> 00:01:05,119
from mit and a phd in computer science

29
00:01:07,350 --> 00:01:06,159
from the university of southern

30
00:01:09,270 --> 00:01:07,360
california

31
00:01:11,030 --> 00:01:09,280
today he leads mission operations

32
00:01:12,390 --> 00:01:11,040
innovation at nasa's jet propulsion

33
00:01:14,950 --> 00:01:12,400
laboratory

34
00:01:17,350 --> 00:01:14,960
he is also the founder and lead of the

35
00:01:19,350 --> 00:01:17,360
jpl operations laboratory and the

36
00:01:21,350 --> 00:01:19,360
principal investigator for a variety of

37
00:01:23,190 --> 00:01:21,360
projects focused on the control of

38
00:01:25,030 --> 00:01:23,200
spacecraft and robots

39
00:01:27,350 --> 00:01:25,040
previously he was a section manager

40
00:01:30,070 --> 00:01:27,360
group supervisor and lead for control

41
00:01:32,870 --> 00:01:30,080
tools on the spirit opportunity and

42
00:01:34,149 --> 00:01:32,880
curiosity mars rover missions ladies and

43
00:01:43,910 --> 00:01:34,159
gentlemen please help me welcome

44
00:01:48,069 --> 00:01:45,990
good evening thank you it's a real

45
00:01:50,550 --> 00:01:48,079
pleasure to come and talk tonight about

46
00:01:52,469 --> 00:01:50,560
a subject of great

47
00:01:55,749 --> 00:01:52,479
personal interest of mine i'm quite

48
00:01:57,270 --> 00:01:55,759
passionate about this concept of using

49
00:02:00,630 --> 00:01:57,280
technologies that are finally coming

50
00:02:02,310 --> 00:02:00,640
together i believe now to enable

51
00:02:05,510 --> 00:02:02,320
not just scientists and engineers to be

52
00:02:07,510 --> 00:02:05,520
better explorers but also to include the

53
00:02:09,990 --> 00:02:07,520
public in the journeys that we're taking

54
00:02:12,150 --> 00:02:10,000
right now within nasa

55
00:02:15,270 --> 00:02:12,160
so the vision that i want to describe to

56
00:02:17,830 --> 00:02:15,280
you today put simply is

57
00:02:19,270 --> 00:02:17,840
based on this

58
00:02:22,630 --> 00:02:19,280
this is how

59
00:02:25,350 --> 00:02:22,640
human beings explore the world

60
00:02:27,350 --> 00:02:25,360
we are all gifted with an amazing innate

61
00:02:29,190 --> 00:02:27,360
ability to extract a great deal of

62
00:02:32,070 --> 00:02:29,200
information from an environment and

63
00:02:33,509 --> 00:02:32,080
understand it just by being present in

64
00:02:35,030 --> 00:02:33,519
it

65
00:02:37,830 --> 00:02:35,040
now scientists and engineers like the

66
00:02:40,150 --> 00:02:37,840
ones that operate our missions build

67
00:02:42,390 --> 00:02:40,160
upon those innate abilities with a lot

68
00:02:44,229 --> 00:02:42,400
of specialized training and learning

69
00:02:44,949 --> 00:02:44,239
until they can read an environment like

70
00:02:46,949 --> 00:02:44,959
this

71
00:02:49,350 --> 00:02:46,959
like a book in fact i've heard

72
00:02:50,949 --> 00:02:49,360
geologists tell me before that every

73
00:02:52,710 --> 00:02:50,959
environment that they go and explore on

74
00:02:54,150 --> 00:02:52,720
earth has a story to tell

75
00:02:55,910 --> 00:02:54,160
and that their

76
00:02:58,150 --> 00:02:55,920
study of that environment is trying to

77
00:03:00,150 --> 00:02:58,160
learn what that story is

78
00:03:03,030 --> 00:03:00,160
that gives me a little bit of a concern

79
00:03:04,470 --> 00:03:03,040
though when you consider that right now

80
00:03:07,110 --> 00:03:04,480
we're asking many of these scientists

81
00:03:09,270 --> 00:03:07,120
and engineers to explore other planets

82
00:03:11,509 --> 00:03:09,280
like this

83
00:03:13,990 --> 00:03:11,519
peering at little pictures

84
00:03:16,470 --> 00:03:14,000
on a flat computer screen

85
00:03:18,710 --> 00:03:16,480
trying to piece together in their minds

86
00:03:19,910 --> 00:03:18,720
a complete model of this distant

87
00:03:21,030 --> 00:03:19,920
environment that they're trying to

88
00:03:22,869 --> 00:03:21,040
explore

89
00:03:26,309 --> 00:03:22,879
understand and study

90
00:03:27,350 --> 00:03:26,319
this is very challenging to accomplish

91
00:03:29,190 --> 00:03:27,360
part of the reason why it's so

92
00:03:30,550 --> 00:03:29,200
challenging is all those wonderful

93
00:03:31,430 --> 00:03:30,560
innate abilities that i just talked

94
00:03:34,070 --> 00:03:31,440
about

95
00:03:36,149 --> 00:03:34,080
are actually working against you here

96
00:03:39,430 --> 00:03:36,159
because you're not accustomed to seeing

97
00:03:41,270 --> 00:03:39,440
the world in this way

98
00:03:43,270 --> 00:03:41,280
the work the problem gets even a little

99
00:03:45,430 --> 00:03:43,280
bit worse when you consider that our

100
00:03:47,990 --> 00:03:45,440
missions are often operated by groups of

101
00:03:49,670 --> 00:03:48,000
scientists and engineers so imagine

102
00:03:52,070 --> 00:03:49,680
all these different people each forming

103
00:03:54,070 --> 00:03:52,080
their own mental model of an environment

104
00:03:55,589 --> 00:03:54,080
each flawed in a different way

105
00:03:57,830 --> 00:03:55,599
and then coming together and trying to

106
00:04:01,270 --> 00:03:57,840
discuss those mental models and come to

107
00:04:03,830 --> 00:04:01,280
a consensus about what we should do next

108
00:04:06,630 --> 00:04:03,840
very challenging

109
00:04:09,190 --> 00:04:06,640
this is what i believe that we need

110
00:04:10,789 --> 00:04:09,200
we need to surround our scientists and

111
00:04:12,470 --> 00:04:10,799
engineers in environments that allow

112
00:04:14,470 --> 00:04:12,480
their innate abilities and their

113
00:04:16,469 --> 00:04:14,480
training and learning

114
00:04:19,030 --> 00:04:16,479
to work in their advantage to help them

115
00:04:20,949 --> 00:04:19,040
understand environments like mars and

116
00:04:22,469 --> 00:04:20,959
other bodies in our solar system and

117
00:04:24,390 --> 00:04:22,479
beyond

118
00:04:25,670 --> 00:04:24,400
we need to immerse them in these

119
00:04:27,590 --> 00:04:25,680
environments

120
00:04:29,110 --> 00:04:27,600
to make them better decision makers and

121
00:04:31,510 --> 00:04:29,120
better explorers

122
00:04:33,510 --> 00:04:31,520
and i can think of a few people

123
00:04:36,390 --> 00:04:33,520
perhaps some folks here today would like

124
00:04:39,510 --> 00:04:36,400
to join them there as well

125
00:04:42,390 --> 00:04:40,870
what's in the way

126
00:04:43,990 --> 00:04:42,400
what's preventing us from building that

127
00:04:45,510 --> 00:04:44,000
right now what's what are the challenges

128
00:04:47,110 --> 00:04:45,520
we have to deal with well there are two

129
00:04:49,510 --> 00:04:47,120
that i'm going to focus on a lot this

130
00:04:51,270 --> 00:04:49,520
evening the first one is pretty basic

131
00:04:53,590 --> 00:04:51,280
it's distance

132
00:04:55,189 --> 00:04:53,600
the places we want to explore are far

133
00:04:56,469 --> 00:04:55,199
away now i'm going to focus a little bit

134
00:04:58,070 --> 00:04:56,479
on mars tonight that doesn't mean that

135
00:04:59,510 --> 00:04:58,080
what i'm talking about doesn't apply to

136
00:05:00,950 --> 00:04:59,520
other planets as well we're just going

137
00:05:03,030 --> 00:05:00,960
to stick with mars

138
00:05:06,950 --> 00:05:03,040
for most of our examples so let's take

139
00:05:09,110 --> 00:05:06,960
mars mars is between 40 and 400 million

140
00:05:10,950 --> 00:05:09,120
kilometers away from earth depending on

141
00:05:13,510 --> 00:05:10,960
the time of year depending on where the

142
00:05:16,390 --> 00:05:13,520
planets are in their orbits

143
00:05:19,350 --> 00:05:16,400
that translates to a one-way signal

144
00:05:22,150 --> 00:05:19,360
delay of 5-20 minutes so

145
00:05:24,390 --> 00:05:22,160
a signal transmitted from earth

146
00:05:25,749 --> 00:05:24,400
from us takes five to 20 minutes to

147
00:05:27,830 --> 00:05:25,759
reach the robot that we're trying to

148
00:05:29,909 --> 00:05:27,840
control on mars

149
00:05:31,909 --> 00:05:29,919
now this seems like

150
00:05:33,510 --> 00:05:31,919
an incredibly challenging problem if

151
00:05:35,990 --> 00:05:33,520
you've ever had a conversation over a

152
00:05:38,070 --> 00:05:36,000
bad phone call overseas then you may

153
00:05:40,070 --> 00:05:38,080
have experienced just a fraction of a

154
00:05:41,590 --> 00:05:40,080
second of time delay and it's irritating

155
00:05:45,350 --> 00:05:41,600
it's hard the conversation starts to

156
00:05:47,909 --> 00:05:45,360
break down this is far longer than that

157
00:05:49,909 --> 00:05:47,919
in fact it seems so long that

158
00:05:52,390 --> 00:05:49,919
at times people have suggested that the

159
00:05:54,469 --> 00:05:52,400
way to overcome it is to just move the

160
00:05:56,790 --> 00:05:54,479
people closer so we'll leave the robot

161
00:05:59,430 --> 00:05:56,800
on mars but we'll put the human in orbit

162
00:06:01,430 --> 00:05:59,440
around mars and so by doing so

163
00:06:04,309 --> 00:06:01,440
we'll have a time delay that's only

164
00:06:06,070 --> 00:06:04,319
going to be around a second or less

165
00:06:07,270 --> 00:06:06,080
i agree this would be a great thing to

166
00:06:09,350 --> 00:06:07,280
have it would certainly make those

167
00:06:11,590 --> 00:06:09,360
robots easier to control

168
00:06:13,510 --> 00:06:11,600
however this would also be

169
00:06:14,870 --> 00:06:13,520
extremely expensive to accomplish and i

170
00:06:18,550 --> 00:06:14,880
believe

171
00:06:20,790 --> 00:06:18,560
unsustainable in the long term

172
00:06:22,870 --> 00:06:20,800
i also don't think it is the best way

173
00:06:25,029 --> 00:06:22,880
that our astronauts can spend their time

174
00:06:27,670 --> 00:06:25,039
contributing to exploration

175
00:06:29,590 --> 00:06:27,680
so let's come back to that five to

176
00:06:31,189 --> 00:06:29,600
twenty minute delay that seemed like

177
00:06:33,430 --> 00:06:31,199
such a barrier

178
00:06:35,909 --> 00:06:33,440
i'm gonna say instead that this seems

179
00:06:38,469 --> 00:06:35,919
like a luxuriously short amount of time

180
00:06:41,510 --> 00:06:38,479
to deal with and here's why because

181
00:06:44,070 --> 00:06:41,520
today when we operate rovers like spirit

182
00:06:45,830 --> 00:06:44,080
opportunity rather and curiosity we're

183
00:06:48,390 --> 00:06:45,840
really dealing with a much longer time

184
00:06:50,390 --> 00:06:48,400
delay and here's why we send commands to

185
00:06:52,150 --> 00:06:50,400
a robot like opportunity typically once

186
00:06:53,189 --> 00:06:52,160
per day in the morning when the robot

187
00:06:55,270 --> 00:06:53,199
wakes up

188
00:06:57,510 --> 00:06:55,280
the robot then spends its whole day

189
00:06:59,510 --> 00:06:57,520
executing all of those commands taking

190
00:07:01,430 --> 00:06:59,520
pictures driving places things like that

191
00:07:03,189 --> 00:07:01,440
and then at the end of its day sends its

192
00:07:05,589 --> 00:07:03,199
information back to us

193
00:07:07,350 --> 00:07:05,599
we then work all night all the march and

194
00:07:10,070 --> 00:07:07,360
night again

195
00:07:11,830 --> 00:07:10,080
to send it a new set of commands on its

196
00:07:14,309 --> 00:07:11,840
following morning and at that point

197
00:07:16,950 --> 00:07:14,319
that's 24 hours later than our last

198
00:07:18,950 --> 00:07:16,960
transmission so i will contend that we

199
00:07:20,469 --> 00:07:18,960
are not in fact dealing with a 5 to 20

200
00:07:22,790 --> 00:07:20,479
minute time delay when we operate a

201
00:07:25,430 --> 00:07:22,800
robot on mars today we're in fact

202
00:07:27,270 --> 00:07:25,440
dealing with a 12 hour time delay

203
00:07:29,830 --> 00:07:27,280
because it's 24 hours between each of

204
00:07:32,309 --> 00:07:29,840
our transmissions

205
00:07:33,830 --> 00:07:32,319
interestingly 12 hours it's about how

206
00:07:36,790 --> 00:07:33,840
long it would take us to bounce our

207
00:07:39,510 --> 00:07:36,800
signal off of pluto

208
00:07:41,430 --> 00:07:39,520
at its furthest distance from the sun

209
00:07:43,350 --> 00:07:41,440
and then back to mars again

210
00:07:45,189 --> 00:07:43,360
so this doesn't seem like the best way

211
00:07:46,629 --> 00:07:45,199
we could imagine operating a robot on

212
00:07:49,029 --> 00:07:46,639
mars

213
00:07:50,950 --> 00:07:49,039
how would we fix this what would we do

214
00:07:52,710 --> 00:07:50,960
to correct this issue so that we could

215
00:07:54,469 --> 00:07:52,720
deal with only five to 20 minutes of

216
00:07:57,029 --> 00:07:54,479
time to like well we know how to fix it

217
00:07:58,950 --> 00:07:57,039
we need something approximating

218
00:08:01,029 --> 00:07:58,960
continuous or near continuous

219
00:08:02,150 --> 00:08:01,039
communication between earth and marsh to

220
00:08:03,909 --> 00:08:02,160
fix this

221
00:08:06,629 --> 00:08:03,919
we already have half the problem pretty

222
00:08:08,950 --> 00:08:06,639
much fixed in that we've already arrayed

223
00:08:10,469 --> 00:08:08,960
the antennas of the deep space network

224
00:08:12,070 --> 00:08:10,479
at three locations around the earth so

225
00:08:13,909 --> 00:08:12,080
that they can talk pretty much anywhere

226
00:08:16,070 --> 00:08:13,919
in space at any time

227
00:08:18,309 --> 00:08:16,080
what we don't have on the other end is a

228
00:08:19,830 --> 00:08:18,319
telecom infrastructure to receive those

229
00:08:23,430 --> 00:08:19,840
signals and pass them on to surface

230
00:08:24,710 --> 00:08:23,440
spacecraft at any time of the day so

231
00:08:26,469 --> 00:08:24,720
we need those

232
00:08:28,230 --> 00:08:26,479
and if we had them

233
00:08:30,309 --> 00:08:28,240
we could send signals pretty much

234
00:08:31,990 --> 00:08:30,319
anytime we wanted to anything in the

235
00:08:34,310 --> 00:08:32,000
martian system

236
00:08:37,990 --> 00:08:34,320
this would be revolutionary for mars

237
00:08:39,909 --> 00:08:38,000
exploration and i would assert that

238
00:08:42,790 --> 00:08:39,919
if we're trying to bring around that

239
00:08:45,110 --> 00:08:42,800
revolution in mars exploration that

240
00:08:47,190 --> 00:08:45,120
a change in our telecom infrastructure

241
00:08:48,550 --> 00:08:47,200
is the first most modest and i think

242
00:08:50,150 --> 00:08:48,560
biggest bang for our buck we could

243
00:08:51,990 --> 00:08:50,160
imagine

244
00:08:53,829 --> 00:08:52,000
why well because it would allow us to

245
00:08:55,269 --> 00:08:53,839
start operating a robot in a very

246
00:08:58,470 --> 00:08:55,279
different way so instead of sending a

247
00:09:00,310 --> 00:08:58,480
full day's worth of commands all at once

248
00:09:01,990 --> 00:09:00,320
we could instead

249
00:09:03,670 --> 00:09:02,000
send a little bit of work for the robot

250
00:09:05,750 --> 00:09:03,680
to accomplish we could say

251
00:09:06,790 --> 00:09:05,760
drive a little bit of a waze

252
00:09:11,110 --> 00:09:06,800
and then

253
00:09:12,949 --> 00:09:11,120
little bit further take some pictures

254
00:09:15,509 --> 00:09:12,959
and tell us what happened again now

255
00:09:17,269 --> 00:09:15,519
because we're able to respond to things

256
00:09:18,870 --> 00:09:17,279
as they're developing meaning we can see

257
00:09:20,310 --> 00:09:18,880
how far it got and change our mind about

258
00:09:22,310 --> 00:09:20,320
what we want to do

259
00:09:24,070 --> 00:09:22,320
this would be a far more effective way

260
00:09:25,670 --> 00:09:24,080
to operate but there is another problem

261
00:09:27,670 --> 00:09:25,680
that i'm hiding in here

262
00:09:30,230 --> 00:09:27,680
this would be good but we can do much

263
00:09:32,230 --> 00:09:30,240
better and the research in my laboratory

264
00:09:34,870 --> 00:09:32,240
has been trying to make this kind of an

265
00:09:37,190 --> 00:09:34,880
operations model even better than this

266
00:09:38,949 --> 00:09:37,200
so here's the problem

267
00:09:40,550 --> 00:09:38,959
when you send that first command to the

268
00:09:42,389 --> 00:09:40,560
rover first of all we know we're already

269
00:09:43,910 --> 00:09:42,399
paying one time delay cost right it

270
00:09:46,150 --> 00:09:43,920
takes 5 to 20 minutes for that signal to

271
00:09:47,990 --> 00:09:46,160
get to mars in the first place

272
00:09:50,710 --> 00:09:48,000
what happens then though this human

273
00:09:52,949 --> 00:09:50,720
being starts to wait because we're

274
00:09:53,910 --> 00:09:52,959
waiting for the robot to do what it was

275
00:09:55,350 --> 00:09:53,920
told

276
00:09:57,030 --> 00:09:55,360
so it

277
00:09:58,949 --> 00:09:57,040
we wait while it completes those

278
00:10:01,269 --> 00:09:58,959
commands we wait for it to send the

279
00:10:03,910 --> 00:10:01,279
results back another 5-20 minutes

280
00:10:05,269 --> 00:10:03,920
and then what happens well now the robot

281
00:10:06,230 --> 00:10:05,279
has to start waiting

282
00:10:07,509 --> 00:10:06,240
because

283
00:10:08,870 --> 00:10:07,519
we have to think about what it's going

284
00:10:10,550 --> 00:10:08,880
to do next

285
00:10:13,030 --> 00:10:10,560
so someone's always waiting in fact when

286
00:10:13,910 --> 00:10:13,040
the signal's in transit both parties are

287
00:10:16,790 --> 00:10:13,920
waiting

288
00:10:18,710 --> 00:10:16,800
so we spend a lot of time with one part

289
00:10:20,069 --> 00:10:18,720
of this very important partnership

290
00:10:21,829 --> 00:10:20,079
waiting

291
00:10:23,590 --> 00:10:21,839
in fact this

292
00:10:25,910 --> 00:10:23,600
method of operations

293
00:10:28,150 --> 00:10:25,920
has been come to known in the time delay

294
00:10:29,509 --> 00:10:28,160
operations world as

295
00:10:31,670 --> 00:10:29,519
bump and weight

296
00:10:33,910 --> 00:10:31,680
because you bump the robot a little bit

297
00:10:35,190 --> 00:10:33,920
and then you wait to see what happened

298
00:10:36,710 --> 00:10:35,200
bump it forward a little bit more and

299
00:10:39,030 --> 00:10:36,720
wait to see what happened how can we get

300
00:10:40,790 --> 00:10:39,040
out of this where could we look for some

301
00:10:42,150 --> 00:10:40,800
inspiration well

302
00:10:44,389 --> 00:10:42,160
turns out

303
00:10:46,230 --> 00:10:44,399
that video games got to deal with a very

304
00:10:47,829 --> 00:10:46,240
similar problem and i'm going to explain

305
00:10:49,990 --> 00:10:47,839
how they deal with it

306
00:10:51,990 --> 00:10:50,000
and then how we're following a

307
00:10:53,030 --> 00:10:52,000
impressively similar result

308
00:10:54,310 --> 00:10:53,040
now

309
00:10:55,750 --> 00:10:54,320
many years ago

310
00:10:57,829 --> 00:10:55,760
some friends of mine and i just for a

311
00:10:59,750 --> 00:10:57,839
hobby wrote a modification to a very

312
00:11:01,509 --> 00:10:59,760
popular video game of the time and this

313
00:11:03,590 --> 00:11:01,519
is actually actual speed it was a very

314
00:11:06,150 --> 00:11:03,600
fast-paced game you know explosions

315
00:11:08,310 --> 00:11:06,160
happening everywhere rockets flying lots

316
00:11:11,190 --> 00:11:08,320
of people running everywhere it was a

317
00:11:12,949 --> 00:11:11,200
bit chaotic and it's very challenging

318
00:11:14,870 --> 00:11:12,959
for a game like this to keep everything

319
00:11:16,630 --> 00:11:14,880
in sync because people are playing from

320
00:11:17,990 --> 00:11:16,640
all over the country and they need to

321
00:11:19,269 --> 00:11:18,000
all kind of have a consistent view of

322
00:11:20,949 --> 00:11:19,279
the world for this game to make any

323
00:11:24,470 --> 00:11:20,959
sense at all now i'm not going to

324
00:11:26,310 --> 00:11:24,480
explain how we resolved it in this game

325
00:11:27,509 --> 00:11:26,320
because it's a bit of a complicated game

326
00:11:29,590 --> 00:11:27,519
i'm going to reduce it to something a

327
00:11:31,430 --> 00:11:29,600
little more simple and perhaps familiar

328
00:11:32,630 --> 00:11:31,440
to some folks in this audience

329
00:11:36,069 --> 00:11:32,640
this

330
00:11:37,509 --> 00:11:36,079
is pitfall harry in the atari 2600 2600

331
00:11:39,110 --> 00:11:37,519
game pitfall

332
00:11:40,870 --> 00:11:39,120
very charming game

333
00:11:42,389 --> 00:11:40,880
played in many many living rooms

334
00:11:45,190 --> 00:11:42,399
including

335
00:11:46,790 --> 00:11:45,200
some that i have been in and the

336
00:11:48,150 --> 00:11:46,800
idea here is you run and jump and try to

337
00:11:49,509 --> 00:11:48,160
keep yourself safe now when you're

338
00:11:51,269 --> 00:11:49,519
playing this by yourself in your living

339
00:11:52,710 --> 00:11:51,279
room it's very easy for things to stay

340
00:11:54,150 --> 00:11:52,720
in sync because there's really only one

341
00:11:55,990 --> 00:11:54,160
version of the world it's the version

342
00:11:57,990 --> 00:11:56,000
that's being run on your atari inside

343
00:11:59,990 --> 00:11:58,000
your your living room so that that's

344
00:12:01,670 --> 00:12:00,000
easy imagine though if we were going to

345
00:12:02,790 --> 00:12:01,680
play a game like this across the

346
00:12:04,550 --> 00:12:02,800
internet

347
00:12:06,790 --> 00:12:04,560
suddenly there's not one version of the

348
00:12:09,269 --> 00:12:06,800
game there's actually two there's one

349
00:12:10,629 --> 00:12:09,279
running on you or your computer and then

350
00:12:12,150 --> 00:12:10,639
there's another one running on the

351
00:12:13,990 --> 00:12:12,160
server and the server's job is a

352
00:12:16,150 --> 00:12:14,000
computer that's responsible for making

353
00:12:17,750 --> 00:12:16,160
sure that your version of the world and

354
00:12:19,990 --> 00:12:17,760
all of the other people who are playing

355
00:12:21,990 --> 00:12:20,000
with you stay consistent and that

356
00:12:23,750 --> 00:12:22,000
certain rules are followed

357
00:12:25,030 --> 00:12:23,760
keeping these versions of the game in

358
00:12:26,470 --> 00:12:25,040
sync

359
00:12:29,269 --> 00:12:26,480
can be very challenging and i'm going to

360
00:12:30,710 --> 00:12:29,279
explain why with a very very simple

361
00:12:32,710 --> 00:12:30,720
example so

362
00:12:33,990 --> 00:12:32,720
simply running and trying to jump over

363
00:12:36,150 --> 00:12:34,000
that pit

364
00:12:37,590 --> 00:12:36,160
let's imagine that you know the server

365
00:12:39,350 --> 00:12:37,600
that's that computer that's responsible

366
00:12:41,910 --> 00:12:39,360
for keeping things in sync between the

367
00:12:43,269 --> 00:12:41,920
players has told us where pitfall harry

368
00:12:45,190 --> 00:12:43,279
is

369
00:12:46,790 --> 00:12:45,200
and we've decided we'd like to run

370
00:12:49,190 --> 00:12:46,800
towards that pit

371
00:12:50,550 --> 00:12:49,200
so we push the key in our computer that

372
00:12:52,470 --> 00:12:50,560
tells it to run

373
00:12:54,710 --> 00:12:52,480
and we have to send a command to the

374
00:12:58,069 --> 00:12:54,720
server telling it that we want to run

375
00:13:00,069 --> 00:12:58,079
because it's actually not okay for our

376
00:13:01,350 --> 00:13:00,079
computer to be completely in charge of

377
00:13:02,790 --> 00:13:01,360
what the world is like because if it

378
00:13:04,870 --> 00:13:02,800
were it would be very easy to cheat you

379
00:13:06,710 --> 00:13:04,880
could simply have your computer say well

380
00:13:08,150 --> 00:13:06,720
i can run 10 times as fast as everyone

381
00:13:10,069 --> 00:13:08,160
jumped 10 times as high and i got all

382
00:13:12,710 --> 00:13:10,079
the points i went right it wouldn't make

383
00:13:14,550 --> 00:13:12,720
it very much fun for the other people so

384
00:13:17,030 --> 00:13:14,560
what we have to do is basically request

385
00:13:18,870 --> 00:13:17,040
of the server that we want to run so

386
00:13:20,470 --> 00:13:18,880
imagine that command starts heading its

387
00:13:23,430 --> 00:13:20,480
way towards the server now the internet

388
00:13:24,790 --> 00:13:23,440
is fast but it is not instantaneous

389
00:13:26,949 --> 00:13:24,800
so

390
00:13:28,949 --> 00:13:26,959
if i if the computer that you're playing

391
00:13:31,430 --> 00:13:28,959
on were to wait around to hear back from

392
00:13:33,430 --> 00:13:31,440
the server saying yes you you're running

393
00:13:35,990 --> 00:13:33,440
then the game would be very stuttery

394
00:13:37,910 --> 00:13:36,000
very jerky very laggy

395
00:13:40,230 --> 00:13:37,920
to play so instead what your computer

396
00:13:42,230 --> 00:13:40,240
has to do is guess it guesses you know

397
00:13:43,509 --> 00:13:42,240
what i'm just going to assume that the

398
00:13:45,190 --> 00:13:43,519
server is going to tell me it's okay so

399
00:13:46,470 --> 00:13:45,200
we're going to start running and you

400
00:13:48,069 --> 00:13:46,480
know what you're about to get into that

401
00:13:49,509 --> 00:13:48,079
pit and you push the button that says

402
00:13:51,430 --> 00:13:49,519
jump you still haven't heard anything

403
00:13:53,030 --> 00:13:51,440
from the server yet and your character

404
00:13:54,949 --> 00:13:53,040
starts jumping and

405
00:13:57,350 --> 00:13:54,959
starts to grab onto that rope and then

406
00:13:59,030 --> 00:13:57,360
finally the server gets back to you

407
00:14:00,550 --> 00:13:59,040
about that command that you said i wanna

408
00:14:02,949 --> 00:14:00,560
i wanna start running

409
00:14:05,430 --> 00:14:02,959
so what are you supposed to do now

410
00:14:07,990 --> 00:14:05,440
how do you stay in sync well what

411
00:14:10,230 --> 00:14:08,000
happens is very very quickly in your

412
00:14:12,150 --> 00:14:10,240
game on your local computer it takes the

413
00:14:14,710 --> 00:14:12,160
information that the server gave it

414
00:14:16,710 --> 00:14:14,720
rushes it back

415
00:14:18,870 --> 00:14:16,720
and says all right what if this is the

416
00:14:20,389 --> 00:14:18,880
way that that run actually turned down

417
00:14:21,750 --> 00:14:20,399
and then it plays forward all the

418
00:14:24,069 --> 00:14:21,760
commands that you

419
00:14:26,870 --> 00:14:24,079
used on your local computer since then

420
00:14:28,470 --> 00:14:26,880
to get a new version of reality and then

421
00:14:29,350 --> 00:14:28,480
when that happens small things might

422
00:14:31,910 --> 00:14:29,360
change

423
00:14:33,030 --> 00:14:31,920
enough

424
00:14:35,350 --> 00:14:33,040
that you don't really notice them when

425
00:14:37,350 --> 00:14:35,360
you're playing a game but your game your

426
00:14:39,189 --> 00:14:37,360
local computer is actually doing this

427
00:14:41,350 --> 00:14:39,199
over and over and over again very low

428
00:14:42,790 --> 00:14:41,360
very quickly locally it's receiving an

429
00:14:45,350 --> 00:14:42,800
updated piece of information from the

430
00:14:48,389 --> 00:14:45,360
server it's running forward from there

431
00:14:50,629 --> 00:14:48,399
to what now is on your computer and

432
00:14:52,150 --> 00:14:50,639
rendering a new version of now over and

433
00:14:53,350 --> 00:14:52,160
over again you just don't realize it's

434
00:15:00,949 --> 00:14:53,360
doing this and of course this is

435
00:15:05,430 --> 00:15:02,790
this dance continues when you're playing

436
00:15:08,470 --> 00:15:05,440
a game and all modern uh online games

437
00:15:10,829 --> 00:15:08,480
work off of a base a system that is like

438
00:15:12,710 --> 00:15:10,839
this now what does this mean for

439
00:15:14,470 --> 00:15:12,720
exploration

440
00:15:16,870 --> 00:15:14,480
well they get to cheat a little bit here

441
00:15:19,509 --> 00:15:16,880
because in order to make the predictions

442
00:15:22,150 --> 00:15:19,519
pretty good they actually run a copy of

443
00:15:24,949 --> 00:15:22,160
the code that the server has

444
00:15:26,310 --> 00:15:24,959
on your local computer so it's kind of

445
00:15:28,470 --> 00:15:26,320
running through the same rule book to

446
00:15:29,910 --> 00:15:28,480
figure out what the server is likely to

447
00:15:31,750 --> 00:15:29,920
predict now i can't predict everything

448
00:15:32,949 --> 00:15:31,760
because for one thing your computer

449
00:15:35,670 --> 00:15:32,959
can't predict the actions of other

450
00:15:37,670 --> 00:15:35,680
players but it can get pretty good

451
00:15:39,110 --> 00:15:37,680
imagine now that what's on the other end

452
00:15:41,030 --> 00:15:39,120
of the time delay

453
00:15:43,509 --> 00:15:41,040
is mars

454
00:15:45,110 --> 00:15:43,519
it's not a computer not a simulation

455
00:15:47,590 --> 00:15:45,120
so we can't run

456
00:15:50,150 --> 00:15:47,600
mars locally we can't run a copy of that

457
00:15:51,509 --> 00:15:50,160
locally so we have to make our best

458
00:15:53,269 --> 00:15:51,519
predictions our best guesses about

459
00:15:55,189 --> 00:15:53,279
what's going to happen but we're going

460
00:15:57,110 --> 00:15:55,199
to be wrong more often we're going to

461
00:15:58,790 --> 00:15:57,120
have more uncertainty

462
00:16:01,030 --> 00:15:58,800
so that comes to

463
00:16:02,550 --> 00:16:01,040
how the research in my lab is trying to

464
00:16:04,790 --> 00:16:02,560
address this particular challenge

465
00:16:06,550 --> 00:16:04,800
because the concept of predicting what

466
00:16:09,269 --> 00:16:06,560
is going to happen in order to mitigate

467
00:16:11,110 --> 00:16:09,279
the effects of time delay is not new but

468
00:16:13,189 --> 00:16:11,120
what we believe is new is providing a

469
00:16:14,470 --> 00:16:13,199
way to reason upon the uncertainty that

470
00:16:16,389 --> 00:16:14,480
that introduces

471
00:16:18,790 --> 00:16:16,399
so a member of our lab alex menzies

472
00:16:20,870 --> 00:16:18,800
actually went so far as to write a video

473
00:16:22,710 --> 00:16:20,880
game version of what it would be like to

474
00:16:24,150 --> 00:16:22,720
drive a robot with time delay and this

475
00:16:25,189 --> 00:16:24,160
is what it looks like so let me explain

476
00:16:27,590 --> 00:16:25,199
a couple of things that you're seeing

477
00:16:29,269 --> 00:16:27,600
here this

478
00:16:31,030 --> 00:16:29,279
black circular robot down here at the

479
00:16:32,870 --> 00:16:31,040
bottom well that is a robot and the red

480
00:16:34,710 --> 00:16:32,880
thing is which way it's facing

481
00:16:36,710 --> 00:16:34,720
and these green dots that are being laid

482
00:16:38,470 --> 00:16:36,720
up here at the top those are waypoints

483
00:16:41,269 --> 00:16:38,480
those are places that you want the robot

484
00:16:42,550 --> 00:16:41,279
to go these dark blue lines here these

485
00:16:45,110 --> 00:16:42,560
are walls these are things that you're

486
00:16:47,749 --> 00:16:45,120
not allowed to bump into

487
00:16:49,590 --> 00:16:47,759
this blue haze that surrounds these

488
00:16:51,430 --> 00:16:49,600
waypoints is what we call the

489
00:16:53,350 --> 00:16:51,440
uncertainty volume

490
00:16:55,829 --> 00:16:53,360
this represents

491
00:16:57,990 --> 00:16:55,839
how unsure you are about where the robot

492
00:17:00,150 --> 00:16:58,000
will actually end up if you tell it to

493
00:17:02,069 --> 00:17:00,160
go to those locations and you'll notice

494
00:17:04,390 --> 00:17:02,079
it grows and shrinks

495
00:17:06,470 --> 00:17:04,400
if you get way out in advance of the

496
00:17:07,750 --> 00:17:06,480
robot which is down here this is where

497
00:17:12,549 --> 00:17:07,760
the robot

498
00:17:14,710 --> 00:17:12,559
was to get too far out in advance that

499
00:17:16,630 --> 00:17:14,720
uncertainty volume gets too big and you

500
00:17:18,949 --> 00:17:16,640
have to stop and wait for it to compress

501
00:17:20,789 --> 00:17:18,959
like right now the driver is waiting

502
00:17:23,669 --> 00:17:20,799
until that blue uncertainty volume gets

503
00:17:24,470 --> 00:17:23,679
small enough that it can fit through

504
00:17:28,470 --> 00:17:24,480
that

505
00:17:30,870 --> 00:17:28,480
passageway there so finally right there

506
00:17:32,870 --> 00:17:30,880
it the driver can see i have enough room

507
00:17:34,870 --> 00:17:32,880
to get through so i will

508
00:17:36,549 --> 00:17:34,880
issue those waypoints and the robot will

509
00:17:37,669 --> 00:17:36,559
start driving again and then he'll start

510
00:17:39,430 --> 00:17:37,679
quickly

511
00:17:41,270 --> 00:17:39,440
adding waypoints to move up to the next

512
00:17:43,110 --> 00:17:41,280
passageway and have to stop again

513
00:17:44,390 --> 00:17:43,120
because now the uncertainty is getting a

514
00:17:46,230 --> 00:17:44,400
little bit too large again like we

515
00:17:47,750 --> 00:17:46,240
really need to wait again and let let

516
00:17:49,430 --> 00:17:47,760
the real robot catch up now why does

517
00:17:50,390 --> 00:17:49,440
that uncertainty volume get smaller when

518
00:17:53,110 --> 00:17:50,400
you wait

519
00:17:54,789 --> 00:17:53,120
it's because as it gets closer to where

520
00:17:56,950 --> 00:17:54,799
you're planning there are fewer and

521
00:17:59,029 --> 00:17:56,960
fewer ways that things can go wrong that

522
00:18:01,510 --> 00:17:59,039
things can be unpredictable because it's

523
00:18:03,029 --> 00:18:01,520
close to to where the wave points are so

524
00:18:04,390 --> 00:18:03,039
it's not going to instantly jump 10 feet

525
00:18:07,669 --> 00:18:04,400
to the left it's going to make small

526
00:18:12,390 --> 00:18:09,669
what we're trying to do is to allow a

527
00:18:15,029 --> 00:18:12,400
person to stay ahead of the robot while

528
00:18:17,029 --> 00:18:15,039
managing risk by understanding the

529
00:18:18,310 --> 00:18:17,039
uncertainty that that introduces there's

530
00:18:20,390 --> 00:18:18,320
one last thing i want to point out which

531
00:18:23,029 --> 00:18:20,400
is this special orange little version of

532
00:18:24,789 --> 00:18:23,039
the robot here that's something we call

533
00:18:29,669 --> 00:18:24,799
the committed state

534
00:18:31,510 --> 00:18:29,679
you think the robot would be if you said

535
00:18:33,350 --> 00:18:31,520
stop everything right now

536
00:18:34,789 --> 00:18:33,360
so you can't stop the robot instantly on

537
00:18:37,669 --> 00:18:34,799
mars because it takes 5 to 20 minutes

538
00:18:39,510 --> 00:18:37,679
for your stop message to get there

539
00:18:41,590 --> 00:18:39,520
where that orange little robot is is

540
00:18:44,070 --> 00:18:41,600
where you would expect it to stop

541
00:18:46,070 --> 00:18:44,080
if that stop command were sent right now

542
00:18:47,510 --> 00:18:46,080
what's also really important about it is

543
00:18:49,350 --> 00:18:47,520
if you can stay

544
00:18:52,230 --> 00:18:49,360
ahead

545
00:18:54,950 --> 00:18:52,240
of that little orange robot

546
00:18:57,110 --> 00:18:54,960
then you're in in fact not paying any

547
00:18:59,190 --> 00:18:57,120
penalty for the time delay because if

548
00:19:01,350 --> 00:18:59,200
you're dropping waypoints in in front of

549
00:19:03,909 --> 00:19:01,360
these in front in front of this orange

550
00:19:06,230 --> 00:19:03,919
robot it means that the robot will get

551
00:19:09,110 --> 00:19:06,240
that instruction before it stops

552
00:19:11,669 --> 00:19:09,120
so this game becomes about can you plan

553
00:19:12,549 --> 00:19:11,679
waypoints in in front of the committed

554
00:19:14,549 --> 00:19:12,559
state

555
00:19:17,669 --> 00:19:14,559
while being very careful never to let

556
00:19:19,590 --> 00:19:17,679
your blue uncertainty volume intersect

557
00:19:22,310 --> 00:19:19,600
with something that you don't want to

558
00:19:24,710 --> 00:19:22,320
come into contact with

559
00:19:26,710 --> 00:19:24,720
if you can do that there's actually no

560
00:19:28,630 --> 00:19:26,720
difference between operating that robot

561
00:19:31,110 --> 00:19:28,640
two feet away or 400 million kilometers

562
00:19:34,710 --> 00:19:32,630
so let's look at how we've tried

563
00:19:37,270 --> 00:19:34,720
applying this to a real robot because

564
00:19:38,789 --> 00:19:37,280
that's just a simulation

565
00:19:42,070 --> 00:19:38,799
this robot

566
00:19:44,549 --> 00:19:42,080
is just a basic off-the-shelf uh

567
00:19:46,870 --> 00:19:44,559
robot that we've outfitted with some

568
00:19:49,510 --> 00:19:46,880
range sensors on top and so those range

569
00:19:50,870 --> 00:19:49,520
sensors allow the robot to keep track of

570
00:19:52,549 --> 00:19:50,880
where things are in the environment in

571
00:19:53,909 --> 00:19:52,559
addition to what they look like and

572
00:19:56,070 --> 00:19:53,919
we've set up a little obstacle course

573
00:19:58,390 --> 00:19:56,080
for it with those red boxes so you can

574
00:20:00,470 --> 00:19:58,400
see the the sensors up top complete with

575
00:20:02,630 --> 00:20:00,480
googly eyes

576
00:20:05,110 --> 00:20:02,640
so this is remember the old way right so

577
00:20:08,150 --> 00:20:05,120
we issue a waypoint and then we wait

578
00:20:09,990 --> 00:20:08,160
until the robot arrives at that spot

579
00:20:11,510 --> 00:20:10,000
and when it gets there and tells us i've

580
00:20:13,110 --> 00:20:11,520
gotten there then we put another

581
00:20:15,270 --> 00:20:13,120
waypoint and then we wait again so

582
00:20:16,630 --> 00:20:15,280
that's bump and weight right

583
00:20:19,029 --> 00:20:16,640
so here is the interface that we've

584
00:20:21,830 --> 00:20:19,039
built that allows us to go beyond that

585
00:20:24,390 --> 00:20:21,840
this is a rendering of the environment

586
00:20:25,909 --> 00:20:24,400
as sensed by those sensors on top of the

587
00:20:28,149 --> 00:20:25,919
robot

588
00:20:30,310 --> 00:20:28,159
and then when we plan a path through

589
00:20:32,870 --> 00:20:30,320
this little obstacle course of red robot

590
00:20:34,470 --> 00:20:32,880
of red boxes you'll see that uncertainty

591
00:20:36,710 --> 00:20:34,480
volume appear

592
00:20:38,710 --> 00:20:36,720
and as we drop additional waypoints we

593
00:20:40,070 --> 00:20:38,720
have to be careful if we ever see the

594
00:20:42,549 --> 00:20:40,080
blue

595
00:20:44,549 --> 00:20:42,559
hazy part come into contact with the box

596
00:20:46,310 --> 00:20:44,559
because that would mean we might crash

597
00:20:48,149 --> 00:20:46,320
we might bump into something so that

598
00:20:49,909 --> 00:20:48,159
situation you see right there is is not

599
00:20:52,230 --> 00:20:49,919
good you'd have to pull that waypoint

600
00:20:53,270 --> 00:20:52,240
back a little bit and that ghosted robot

601
00:20:54,870 --> 00:20:53,280
you saw on the interface and we're

602
00:20:56,710 --> 00:20:54,880
giving you just an idea of what it would

603
00:20:58,310 --> 00:20:56,720
look like if it existed in real life

604
00:21:00,470 --> 00:20:58,320
that's that magical state called the

605
00:21:02,390 --> 00:21:00,480
committed state that if you're planning

606
00:21:05,029 --> 00:21:02,400
in front of that you're not paying any

607
00:21:06,789 --> 00:21:05,039
penalty for time delay

608
00:21:08,789 --> 00:21:06,799
so we've had some encouraging results

609
00:21:10,630 --> 00:21:08,799
with this interface we're about to start

610
00:21:12,870 --> 00:21:10,640
some experiments where we hope we'll be

611
00:21:14,630 --> 00:21:12,880
able to show some quantitative benefits

612
00:21:15,590 --> 00:21:14,640
to people who choose to drive robots

613
00:21:21,110 --> 00:21:15,600
this way

614
00:21:27,510 --> 00:21:23,590
what would it enable if we could drive a

615
00:21:29,110 --> 00:21:27,520
robot that way on mars

616
00:21:30,710 --> 00:21:29,120
well saw what it looked like with bump

617
00:21:33,270 --> 00:21:30,720
and weight you saw what it looked like

618
00:21:34,950 --> 00:21:33,280
when you just kept uh when you had

619
00:21:36,390 --> 00:21:34,960
shorter bump in weights right when you

620
00:21:38,070 --> 00:21:36,400
gave it just a few things to do and then

621
00:21:39,350 --> 00:21:38,080
a few more things to do

622
00:21:41,510 --> 00:21:39,360
if we could drive it the way that i just

623
00:21:43,669 --> 00:21:41,520
showed you would be more like this we

624
00:21:45,270 --> 00:21:43,679
would send commands to tell the robot to

625
00:21:46,950 --> 00:21:45,280
move it would be

626
00:21:49,270 --> 00:21:46,960
reporting back to us about how things

627
00:21:51,909 --> 00:21:49,280
are happening and we would be passing

628
00:21:53,510 --> 00:21:51,919
out new waypoints to it kind of right in

629
00:21:55,510 --> 00:21:53,520
front of it

630
00:21:57,029 --> 00:21:55,520
far enough out where we're managing our

631
00:21:59,270 --> 00:21:57,039
uncertainty so that we're not doing

632
00:22:01,029 --> 00:21:59,280
anything dangerous

633
00:22:02,789 --> 00:22:01,039
but still we're trying to keep the robot

634
00:22:04,390 --> 00:22:02,799
from sitting idle and waiting for a

635
00:22:06,549 --> 00:22:04,400
command

636
00:22:09,430 --> 00:22:06,559
this would be a dramatic benefit to

637
00:22:11,430 --> 00:22:09,440
operations and to exploration because

638
00:22:12,950 --> 00:22:11,440
it would allow us to not only

639
00:22:14,230 --> 00:22:12,960
react quickly to things as they're

640
00:22:15,430 --> 00:22:14,240
changing meaning that you could drive a

641
00:22:17,110 --> 00:22:15,440
little bit see what you see and then

642
00:22:18,870 --> 00:22:17,120
choose where to drive or take a big

643
00:22:20,470 --> 00:22:18,880
picture at low resolution and then pick

644
00:22:22,630 --> 00:22:20,480
an area to take a higher resolution

645
00:22:24,549 --> 00:22:22,640
interactively right then

646
00:22:26,870 --> 00:22:24,559
but

647
00:22:28,549 --> 00:22:26,880
it would also allow us to manage the

648
00:22:32,549 --> 00:22:28,559
risks associated with getting too far

649
00:22:37,430 --> 00:22:35,510
so we'll go on to something else now

650
00:22:40,070 --> 00:22:37,440
distance was one of the barriers in

651
00:22:42,310 --> 00:22:40,080
front of the tele exploration project

652
00:22:44,149 --> 00:22:42,320
immersion is another challenge what do

653
00:22:45,350 --> 00:22:44,159
we do with the information that comes

654
00:22:47,110 --> 00:22:45,360
back from these robots when we're

655
00:22:48,630 --> 00:22:47,120
controlling them how do we present it to

656
00:22:50,630 --> 00:22:48,640
the people who are responsible for

657
00:22:52,549 --> 00:22:50,640
driving them so that they have an

658
00:22:54,470 --> 00:22:52,559
intuitive understanding of the

659
00:22:58,950 --> 00:22:54,480
environment of the robot and can make

660
00:23:03,669 --> 00:23:02,470
imagine yourself as a mars scientist

661
00:23:05,990 --> 00:23:03,679
who's trying to help control the

662
00:23:10,070 --> 00:23:06,000
curiosity rover you are inundated with a

663
00:23:11,590 --> 00:23:10,080
torrent of imagery and other data

664
00:23:14,549 --> 00:23:11,600
you're trying to put together a mental

665
00:23:15,990 --> 00:23:14,559
model of the scene around the robot

666
00:23:18,630 --> 00:23:16,000
we've come up with some tools to help

667
00:23:20,230 --> 00:23:18,640
ourselves do this we build maps

668
00:23:22,230 --> 00:23:20,240
we take the pictures and we stitch them

669
00:23:23,909 --> 00:23:22,240
together into larger pictures and then

670
00:23:25,990 --> 00:23:23,919
we reproject them from interesting

671
00:23:29,510 --> 00:23:26,000
angles to try to get it all in our head

672
00:23:32,390 --> 00:23:29,520
about what the scene really looks like

673
00:23:34,950 --> 00:23:32,400
sometimes we put the robot

674
00:23:36,310 --> 00:23:34,960
as a 3d model into a scene in a computer

675
00:23:38,390 --> 00:23:36,320
and then fly around it to try to

676
00:23:39,909 --> 00:23:38,400
understand where the terrain is and how

677
00:23:41,909 --> 00:23:39,919
big the robot is relative to some of the

678
00:23:44,230 --> 00:23:41,919
things in it but we spend quite a lot of

679
00:23:46,310 --> 00:23:44,240
time looking at images like this this is

680
00:23:48,710 --> 00:23:46,320
a panorama one of my favorites acquired

681
00:23:50,390 --> 00:23:48,720
by the spirit rover around a thousand

682
00:23:52,549 --> 00:23:50,400
days into its mission

683
00:23:54,230 --> 00:23:52,559
i want you to study with it study it

684
00:23:55,510 --> 00:23:54,240
with me for a moment

685
00:23:57,990 --> 00:23:55,520
try to get a feeling for this

686
00:23:59,830 --> 00:23:58,000
environment that the rover is exploring

687
00:24:02,549 --> 00:23:59,840
now try to imagine yourself as an

688
00:24:04,470 --> 00:24:02,559
astronaut there with it

689
00:24:06,230 --> 00:24:04,480
perhaps you've come over here to study a

690
00:24:07,909 --> 00:24:06,240
rock that looks interesting over there

691
00:24:09,830 --> 00:24:07,919
and you decide that you'd like to go to

692
00:24:11,190 --> 00:24:09,840
another place perhaps over there where

693
00:24:12,950 --> 00:24:11,200
there's a patch of soil that looks

694
00:24:14,630 --> 00:24:12,960
interesting

695
00:24:17,029 --> 00:24:14,640
look at this picture and tell me how you

696
00:24:19,269 --> 00:24:17,039
would get there

697
00:24:21,430 --> 00:24:19,279
it seems reasonable that you would just

698
00:24:23,669 --> 00:24:21,440
follow that path

699
00:24:24,870 --> 00:24:23,679
seems very reasonable but incorrect

700
00:24:26,470 --> 00:24:24,880
because if you were there people would

701
00:24:28,070 --> 00:24:26,480
probably start laughing at you because

702
00:24:29,430 --> 00:24:28,080
you would have turned around and walked

703
00:24:31,909 --> 00:24:29,440
directly away from where you wanted to

704
00:24:34,230 --> 00:24:31,919
go because this is a panoramic image

705
00:24:36,230 --> 00:24:34,240
this is the fastest way to get from

706
00:24:37,990 --> 00:24:36,240
point a to point b

707
00:24:40,470 --> 00:24:38,000
much faster in fact those spots are only

708
00:24:42,070 --> 00:24:40,480
a few feet apart

709
00:24:44,070 --> 00:24:42,080
and although those little silhouettes

710
00:24:45,590 --> 00:24:44,080
look kind of reasonable right now if you

711
00:24:46,950 --> 00:24:45,600
really were standing there on mars

712
00:24:48,549 --> 00:24:46,960
studying that rock

713
00:24:49,990 --> 00:24:48,559
you wouldn't be that big you'd be that

714
00:24:51,750 --> 00:24:50,000
big

715
00:24:54,230 --> 00:24:51,760
and unless you grew to about 20 feet

716
00:24:56,470 --> 00:24:54,240
tall while you walked you'd be that big

717
00:24:58,070 --> 00:24:56,480
over on the other side

718
00:25:00,710 --> 00:24:58,080
i tried very hard to make these

719
00:25:03,269 --> 00:25:00,720
measurements it's even hard to

720
00:25:04,789 --> 00:25:03,279
produce an accurate size when you're

721
00:25:06,630 --> 00:25:04,799
working with data that's only rendered

722
00:25:07,990 --> 00:25:06,640
this way now when you first looked at

723
00:25:09,750 --> 00:25:08,000
that image

724
00:25:11,669 --> 00:25:09,760
you probably didn't see anything that

725
00:25:13,350 --> 00:25:11,679
looked peculiar it probably looked

726
00:25:14,950 --> 00:25:13,360
normal probably looked like something

727
00:25:17,750 --> 00:25:14,960
you would just see outside your window

728
00:25:19,029 --> 00:25:17,760
if you were sitting there on mars

729
00:25:20,470 --> 00:25:19,039
if i had shown you a picture of

730
00:25:21,350 --> 00:25:20,480
something a little more familiar like

731
00:25:24,070 --> 00:25:21,360
this

732
00:25:26,390 --> 00:25:24,080
you which by the way is the exact same

733
00:25:27,590 --> 00:25:26,400
kind of image

734
00:25:29,110 --> 00:25:27,600
you would have instantly noticed that

735
00:25:31,029 --> 00:25:29,120
there was something a little bit odd

736
00:25:33,830 --> 00:25:31,039
because you've seen cities

737
00:25:35,909 --> 00:25:33,840
and they don't usually look like this

738
00:25:36,950 --> 00:25:35,919
now let me ask you a question a little

739
00:25:39,990 --> 00:25:36,960
quiz

740
00:25:41,590 --> 00:25:40,000
if you're there and you want to go there

741
00:25:42,950 --> 00:25:41,600
how would you do it and i'm going to

742
00:25:45,110 --> 00:25:42,960
pause for a second let you think about

743
00:25:47,110 --> 00:25:45,120
it how do you get from the arrow on the

744
00:25:48,230 --> 00:25:47,120
right to the arrow that's kind of in the

745
00:25:49,669 --> 00:25:48,240
middle there

746
00:25:51,590 --> 00:25:49,679
what's the path it would you would take

747
00:25:55,110 --> 00:25:51,600
where would you have to turn

748
00:25:59,350 --> 00:25:57,029
okay it's a trick question you would

749
00:26:01,830 --> 00:25:59,360
walk in a straight line it is on the

750
00:26:02,870 --> 00:26:01,840
same street you would walk directly down

751
00:26:04,789 --> 00:26:02,880
the street

752
00:26:06,789 --> 00:26:04,799
not turn at all and you would arrive

753
00:26:08,149 --> 00:26:06,799
immediately at your destination

754
00:26:10,630 --> 00:26:08,159
and if you were there you would

755
00:26:12,149 --> 00:26:10,640
certainly do just that

756
00:26:13,110 --> 00:26:12,159
how about an inside environment like

757
00:26:16,149 --> 00:26:13,120
this

758
00:26:18,470 --> 00:26:16,159
assembly room if you want to walk from

759
00:26:21,269 --> 00:26:18,480
that aisle over to that aisle no it's

760
00:26:24,230 --> 00:26:21,279
the same aisle same trick right

761
00:26:26,070 --> 00:26:24,240
it is really if you can see the footpath

762
00:26:28,149 --> 00:26:26,080
down across the bottom there see that

763
00:26:33,190 --> 00:26:28,159
little curve

764
00:26:37,110 --> 00:26:34,710
when we see things that are familiar to

765
00:26:38,789 --> 00:26:37,120
us projected this way we immediately

766
00:26:40,470 --> 00:26:38,799
recognize that they're not an accurate

767
00:26:43,110 --> 00:26:40,480
representation of the way the world

768
00:26:45,110 --> 00:26:43,120
really is they're still challenging to

769
00:26:47,269 --> 00:26:45,120
interpret even when we do know even if

770
00:26:48,310 --> 00:26:47,279
you know that this room actually looks

771
00:26:50,390 --> 00:26:48,320
like that

772
00:26:53,430 --> 00:26:50,400
it's still really hard to try to map

773
00:26:55,190 --> 00:26:53,440
those things to each other to each other

774
00:26:56,789 --> 00:26:55,200
it gets even worse if things are moving

775
00:26:57,909 --> 00:26:56,799
good thing we don't do extreme sports

776
00:26:59,990 --> 00:26:57,919
this way right

777
00:27:02,549 --> 00:27:00,000
watch your friend on the red jacket here

778
00:27:07,669 --> 00:27:02,559
does that make sense

779
00:27:11,430 --> 00:27:09,350
that's pretty peculiar

780
00:27:13,909 --> 00:27:11,440
what's going on here

781
00:27:15,990 --> 00:27:13,919
the bottom line is is that we are not

782
00:27:17,590 --> 00:27:16,000
evolved or taught

783
00:27:18,549 --> 00:27:17,600
to view and understand the world this

784
00:27:21,269 --> 00:27:18,559
way

785
00:27:23,990 --> 00:27:21,279
and even with great mental effort

786
00:27:26,149 --> 00:27:24,000
we cannot enforce a model of the world

787
00:27:27,590 --> 00:27:26,159
that makes sense on the data that looks

788
00:27:30,870 --> 00:27:27,600
like that

789
00:27:35,269 --> 00:27:33,350
so what are we going to do about it

790
00:27:37,269 --> 00:27:35,279
how can we show things to people what

791
00:27:38,870 --> 00:27:37,279
kind of an environment can you put them

792
00:27:40,310 --> 00:27:38,880
in that would make them feel a little

793
00:27:43,190 --> 00:27:40,320
bit more like they're there who could we

794
00:27:46,630 --> 00:27:45,350
video games again

795
00:27:49,350 --> 00:27:46,640
so i wish i could tell you that this was

796
00:27:50,870 --> 00:27:49,360
a video of me you know out snowboarding

797
00:27:52,310 --> 00:27:50,880
although

798
00:27:54,950 --> 00:27:52,320
i hope there's no one in this audience

799
00:27:56,549 --> 00:27:54,960
that snowboards this way

800
00:27:57,909 --> 00:27:56,559
i've forgotten the name of this this

801
00:27:59,669 --> 00:27:57,919
snowboarding game but i put it here to

802
00:28:01,190 --> 00:27:59,679
make a point which is that video game

803
00:28:02,950 --> 00:28:01,200
industry

804
00:28:05,190 --> 00:28:02,960
is all about immersion

805
00:28:06,950 --> 00:28:05,200
it's about trying to make you feel like

806
00:28:09,110 --> 00:28:06,960
you're in a place that either doesn't

807
00:28:11,110 --> 00:28:09,120
exist or a place that you would never go

808
00:28:12,950 --> 00:28:11,120
in your right mind

809
00:28:15,990 --> 00:28:12,960
and give imbue you with powers that you

810
00:28:18,549 --> 00:28:16,000
would never have in those environments

811
00:28:22,149 --> 00:28:18,559
i'll also point out that they build

812
00:28:24,389 --> 00:28:22,159
immersion with a budget that far exceeds

813
00:28:27,990 --> 00:28:24,399
all of the world's space exploration

814
00:28:31,750 --> 00:28:28,830
no

815
00:28:33,510 --> 00:28:31,760
jokes so

816
00:28:35,750 --> 00:28:33,520
they don't only do it with

817
00:28:37,510 --> 00:28:35,760
cool renderings like that they also do

818
00:28:39,669 --> 00:28:37,520
it with amazing devices

819
00:28:40,549 --> 00:28:39,679
so here are just a few examples

820
00:28:46,070 --> 00:28:40,559
of

821
00:28:47,190 --> 00:28:46,080
many millions into usability and human

822
00:28:49,110 --> 00:28:47,200
interaction

823
00:28:51,590 --> 00:28:49,120
controllers that sort of just beg to be

824
00:28:53,909 --> 00:28:51,600
held head mounted displays body tracking

825
00:28:56,389 --> 00:28:53,919
systems motion tracking systems these

826
00:28:59,750 --> 00:28:56,399
are all tools to try to make you feel

827
00:29:01,750 --> 00:28:59,760
more like you're in the game

828
00:29:02,549 --> 00:29:01,760
okay so you may be looking at this and

829
00:29:03,590 --> 00:29:02,559
saying

830
00:29:04,789 --> 00:29:03,600
gimmicks

831
00:29:06,710 --> 00:29:04,799
these don't really make that big of a

832
00:29:08,230 --> 00:29:06,720
difference well i'm going to make a

833
00:29:09,909 --> 00:29:08,240
pretty impassioned argument to the

834
00:29:12,070 --> 00:29:09,919
contrary starting with just a very

835
00:29:14,149 --> 00:29:12,080
simple example

836
00:29:15,510 --> 00:29:14,159
if you don't like roller coasters you

837
00:29:17,430 --> 00:29:15,520
might want to close your eyes for just a

838
00:29:18,870 --> 00:29:17,440
second because we're going to go on a

839
00:29:20,310 --> 00:29:18,880
little bit of a ride

840
00:29:22,549 --> 00:29:20,320
there's nothing really magical here

841
00:29:24,789 --> 00:29:22,559
we've just stuck a camera on the front

842
00:29:27,110 --> 00:29:24,799
of using a roller coaster here and we're

843
00:29:28,789 --> 00:29:27,120
going to see what it feels like to go

844
00:29:30,470 --> 00:29:28,799
down this now

845
00:29:33,590 --> 00:29:30,480
all right here we go

846
00:29:33,600 --> 00:29:37,909
do you feel something

847
00:29:42,950 --> 00:29:39,669
do you feel

848
00:29:46,230 --> 00:29:44,470
why

849
00:29:47,909 --> 00:29:46,240
you're sitting in an auditorium you're

850
00:29:49,269 --> 00:29:47,919
looking at a screen that's 30 40 feet

851
00:29:51,269 --> 00:29:49,279
away from some of you

852
00:29:52,389 --> 00:29:51,279
why does it feel like you might be

853
00:29:53,750 --> 00:29:52,399
moving

854
00:29:55,190 --> 00:29:53,760
why did it feel more like you were

855
00:29:56,230 --> 00:29:55,200
moving when i made it just a little bit

856
00:29:58,389 --> 00:29:56,240
bigger

857
00:30:00,470 --> 00:29:58,399
this is first in a number of examples

858
00:30:01,269 --> 00:30:00,480
i'm going to give that i would say are

859
00:30:03,750 --> 00:30:01,279
just

860
00:30:05,430 --> 00:30:03,760
sort of a little bit of a funny proof of

861
00:30:07,190 --> 00:30:05,440
the fact that we are a little more

862
00:30:09,269 --> 00:30:07,200
simple than any of us really want to

863
00:30:11,350 --> 00:30:09,279
believe

864
00:30:12,470 --> 00:30:11,360
just by covering a little bit more of

865
00:30:14,230 --> 00:30:12,480
your retina

866
00:30:15,909 --> 00:30:14,240
and making things go rush past you like

867
00:30:20,070 --> 00:30:15,919
that you get a little feeling in your

868
00:30:24,549 --> 00:30:22,149
now let's take this a step further what

869
00:30:26,470 --> 00:30:24,559
if you take a person who's a little bit

870
00:30:28,230 --> 00:30:26,480
afraid of roller coasters

871
00:30:30,310 --> 00:30:28,240
and you put them inside a head-mounted

872
00:30:32,230 --> 00:30:30,320
display

873
00:30:43,950 --> 00:30:32,240
in which they ride a roller coaster

874
00:30:43,960 --> 00:31:01,990
oh my gosh

875
00:31:06,310 --> 00:31:03,430
i don't want to play anymore he just

876
00:31:06,320 --> 00:31:13,110
it's great to have friends

877
00:31:13,120 --> 00:31:15,669
listen

878
00:31:20,310 --> 00:31:17,430
get this off again

879
00:31:22,230 --> 00:31:20,320
he's hyperventilating at the end okay

880
00:31:23,669 --> 00:31:22,240
so you know we can wonder whether it was

881
00:31:25,350 --> 00:31:23,679
really particularly kind of his friends

882
00:31:27,510 --> 00:31:25,360
to put him through that but let's take a

883
00:31:29,350 --> 00:31:27,520
look here these are grown adults

884
00:31:30,789 --> 00:31:29,360
they know where they are they know that

885
00:31:32,230 --> 00:31:30,799
they're sitting in a chair in their home

886
00:31:34,470 --> 00:31:32,240
they know that just 30 seconds ago they

887
00:31:36,389 --> 00:31:34,480
were talking with their friends

888
00:31:37,990 --> 00:31:36,399
why are they reacting this way

889
00:31:40,389 --> 00:31:38,000
why are they scared of something that

890
00:31:41,990 --> 00:31:40,399
they know isn't a rollercoaster

891
00:31:43,590 --> 00:31:42,000
again i think it's because we're a

892
00:31:45,430 --> 00:31:43,600
little more simple than any of us want

893
00:31:47,269 --> 00:31:45,440
to really think that we are

894
00:31:49,909 --> 00:31:47,279
let's look at one other example

895
00:31:51,269 --> 00:31:49,919
continuing with the theme of terror

896
00:31:54,070 --> 00:31:51,279
here is a

897
00:31:55,830 --> 00:31:54,080
scary game played by more grown adults

898
00:31:57,430 --> 00:31:55,840
who are used to scary movies and scary

899
00:32:10,870 --> 00:31:57,440
video games all is going to happen is

900
00:32:16,149 --> 00:32:12,950
i don't want to be

901
00:32:17,509 --> 00:32:16,159
anymore man

902
00:32:29,350 --> 00:32:17,519
i don't want to be in here anymore is

903
00:32:29,360 --> 00:32:34,870
oh my god

904
00:32:40,230 --> 00:32:37,590
no no no no no no no no no no

905
00:32:42,470 --> 00:32:40,240
no no no no no no no no

906
00:32:45,750 --> 00:32:42,480
no no no no no

907
00:32:58,870 --> 00:32:48,470
okay

908
00:33:01,269 --> 00:32:58,880
oh i'm done all right these

909
00:33:03,669 --> 00:33:01,279
i love these examples

910
00:33:06,230 --> 00:33:03,679
how can grown men

911
00:33:07,509 --> 00:33:06,240
be terrified of a video game like this

912
00:33:09,669 --> 00:33:07,519
well

913
00:33:11,350 --> 00:33:09,679
i incidentally this last person here

914
00:33:13,830 --> 00:33:11,360
wrote in his comments on his youtube

915
00:33:15,990 --> 00:33:13,840
video that this was the most terrifying

916
00:33:17,590 --> 00:33:16,000
video game movie book whatever

917
00:33:19,509 --> 00:33:17,600
experience that he'd ever had in his

918
00:33:20,870 --> 00:33:19,519
life

919
00:33:22,630 --> 00:33:20,880
there's some critical things that are

920
00:33:25,190 --> 00:33:22,640
happening here complete

921
00:33:26,470 --> 00:33:25,200
isolation of the visual field of the

922
00:33:27,430 --> 00:33:26,480
player

923
00:33:28,950 --> 00:33:27,440
audio

924
00:33:33,430 --> 00:33:28,960
three-dimensional isolation of their

925
00:33:38,630 --> 00:33:36,310
there are factors here working together

926
00:33:39,830 --> 00:33:38,640
to make these people willing to accept

927
00:33:42,149 --> 00:33:39,840
the environment that they're being

928
00:33:44,230 --> 00:33:42,159
presented with this is what immersion

929
00:33:47,430 --> 00:33:44,240
looks like and here it's being used for

930
00:33:51,509 --> 00:33:47,440
entertainment to scare to shock

931
00:33:53,509 --> 00:33:51,519
how can we use it for exploration

932
00:33:56,630 --> 00:33:53,519
i'm going to contend that when you

933
00:33:58,950 --> 00:33:56,640
accomplish this kind of immersion

934
00:34:00,950 --> 00:33:58,960
you are making it possible for humans to

935
00:34:02,549 --> 00:34:00,960
use those innate abilities that i was

936
00:34:04,830 --> 00:34:02,559
talking about at the beginning of the

937
00:34:10,869 --> 00:34:04,840
talk so i'm going to give

938
00:34:23,270 --> 00:34:13,190
of how my lab is using these kinds of

939
00:34:27,750 --> 00:34:24,869
okay

940
00:34:29,829 --> 00:34:27,760
so what i have here is an oculus rift it

941
00:34:31,270 --> 00:34:29,839
is a head mounted display

942
00:34:33,190 --> 00:34:31,280
in active development right now by a

943
00:34:35,109 --> 00:34:33,200
team in long beach we've been partnering

944
00:34:36,069 --> 00:34:35,119
with them for some time

945
00:34:37,829 --> 00:34:36,079
may have seen some of them on the

946
00:34:39,990 --> 00:34:37,839
internet

947
00:34:42,230 --> 00:34:40,000
it's a couple of screens placed very

948
00:34:43,829 --> 00:34:42,240
close to your eyes with optics in front

949
00:34:46,230 --> 00:34:43,839
of them

950
00:34:48,230 --> 00:34:46,240
and a device in here that is keeping

951
00:34:50,790 --> 00:34:48,240
track of which way it's facing so as i

952
00:34:52,790 --> 00:34:50,800
turn it i've got a scene

953
00:34:55,190 --> 00:34:52,800
that was taken shortly after curiosity's

954
00:34:56,310 --> 00:34:55,200
landing up there so if i put this on my

955
00:34:58,950 --> 00:34:56,320
face here

956
00:35:00,150 --> 00:34:58,960
i also see it in stereo

957
00:35:01,910 --> 00:35:00,160
so that means that i'm getting a

958
00:35:03,990 --> 00:35:01,920
different version on my left eye than my

959
00:35:05,510 --> 00:35:04,000
right eye and it helps me to sense how

960
00:35:07,349 --> 00:35:05,520
far things are away

961
00:35:08,550 --> 00:35:07,359
i can actually go to a few other places

962
00:35:11,430 --> 00:35:08,560
on mars here

963
00:35:13,030 --> 00:35:11,440
and i want to talk about what happens

964
00:35:14,230 --> 00:35:13,040
in a system like this

965
00:35:18,069 --> 00:35:14,240
because

966
00:35:23,430 --> 00:35:19,270
is

967
00:35:25,670 --> 00:35:23,440
on the screens

968
00:35:27,589 --> 00:35:25,680
this sense of which way i'm turning my

969
00:35:30,150 --> 00:35:27,599
head it's often called the kinesthetic

970
00:35:31,670 --> 00:35:30,160
or somatosensory sense it's the part of

971
00:35:33,349 --> 00:35:31,680
our brain that's responsible for knowing

972
00:35:36,150 --> 00:35:33,359
where our body is

973
00:35:38,310 --> 00:35:36,160
when you synchronize that sense with

974
00:35:40,870 --> 00:35:38,320
what's coming into your eyes

975
00:35:42,630 --> 00:35:40,880
your brain immediately takes a jump in

976
00:35:44,550 --> 00:35:42,640
how it can put an environment like this

977
00:35:48,550 --> 00:35:44,560
together

978
00:35:52,150 --> 00:35:49,829
i want to show you one thing it doesn't

979
00:35:53,510 --> 00:35:52,160
do remember this because i'll

980
00:35:54,710 --> 00:35:53,520
give you an example later about how

981
00:35:56,710 --> 00:35:54,720
we're dealing with it

982
00:36:00,390 --> 00:35:56,720
so i can turn it keeps track of which

983
00:36:02,150 --> 00:36:00,400
way it's facing but if i slide it

984
00:36:03,829 --> 00:36:02,160
hard to hold it still enough

985
00:36:05,910 --> 00:36:03,839
nothing changes

986
00:36:07,190 --> 00:36:05,920
so it's not actually keeping track of

987
00:36:10,829 --> 00:36:07,200
its position

988
00:36:14,150 --> 00:36:10,839
it's keeping track of its orientation

989
00:36:16,470 --> 00:36:14,160
okay what's the next step

990
00:36:18,150 --> 00:36:16,480
we've got

991
00:36:21,750 --> 00:36:18,160
our eyes

992
00:36:28,390 --> 00:36:22,790
neck

993
00:36:33,589 --> 00:36:29,990
but we're not able to move around in the

994
00:36:36,630 --> 00:36:33,599
environment we're only able to

995
00:36:37,910 --> 00:36:36,640
control which way we're facing

996
00:36:40,310 --> 00:36:37,920
so the next demonstration i'm going to

997
00:36:41,750 --> 00:36:40,320
show you

998
00:36:43,750 --> 00:36:41,760
allows us to start moving around

999
00:36:48,150 --> 00:36:43,760
although in a basic way in a different

1000
00:36:53,430 --> 00:36:50,150
okay have a second for this data to load

1001
00:36:55,109 --> 00:36:53,440
in the data that you're going to see

1002
00:36:56,230 --> 00:36:55,119
here

1003
00:36:57,510 --> 00:36:56,240
is all acquired from the mars

1004
00:36:58,870 --> 00:36:57,520
reconnaissance orbiter so it's going to

1005
00:37:00,390 --> 00:36:58,880
look a little bit blurry but it's real

1006
00:37:01,750 --> 00:37:00,400
data of marsh

1007
00:37:03,109 --> 00:37:01,760
and i'm going to put on this headset and

1008
00:37:09,589 --> 00:37:03,119
i'm going to really put it on this time

1009
00:37:13,990 --> 00:37:11,510
the way i'm going to move around is with

1010
00:37:15,430 --> 00:37:14,000
this joystick this game pad so i can

1011
00:37:16,790 --> 00:37:15,440
look

1012
00:37:18,710 --> 00:37:16,800
and see things in different directions

1013
00:37:20,550 --> 00:37:18,720
just like i could before but now i can

1014
00:37:22,790 --> 00:37:20,560
also move

1015
00:37:24,150 --> 00:37:22,800
so i'm going to move over here to

1016
00:37:26,550 --> 00:37:24,160
where we have the curiosity rover

1017
00:37:28,870 --> 00:37:26,560
waiting for me

1018
00:37:30,710 --> 00:37:28,880
looking around it's like a nice place

1019
00:37:31,670 --> 00:37:30,720
this is in gale crater incidentally this

1020
00:37:33,109 --> 00:37:31,680
is in

1021
00:37:35,030 --> 00:37:33,119
curiosity's landing site although it's

1022
00:37:37,910 --> 00:37:35,040
not currently located right here

1023
00:37:39,510 --> 00:37:37,920
and i can look around this robot now in

1024
00:37:41,990 --> 00:37:39,520
different directions from different

1025
00:37:43,829 --> 00:37:42,000
directions i'm using this gamepad here

1026
00:37:45,510 --> 00:37:43,839
to pick where i'm standing

1027
00:37:48,829 --> 00:37:45,520
and then i'm using

1028
00:37:53,270 --> 00:37:51,750
checking to decide where i'm looking so

1029
00:37:55,510 --> 00:37:53,280
i can you know get up here and take a

1030
00:37:57,109 --> 00:37:55,520
closer look at the mast

1031
00:37:59,670 --> 00:37:57,119
now what happened there you see me

1032
00:38:01,990 --> 00:37:59,680
trying to slide my head again

1033
00:38:03,270 --> 00:38:02,000
nothing's happening right because again

1034
00:38:04,470 --> 00:38:03,280
it's tracking the direction that i'm

1035
00:38:06,230 --> 00:38:04,480
looking

1036
00:38:08,230 --> 00:38:06,240
but not the position the only position

1037
00:38:10,230 --> 00:38:08,240
control i have is here there's another

1038
00:38:12,710 --> 00:38:10,240
problem though which is that

1039
00:38:15,670 --> 00:38:12,720
you and i do not move through the world

1040
00:38:16,829 --> 00:38:15,680
using a thumb stick

1041
00:38:18,870 --> 00:38:16,839
we

1042
00:38:21,349 --> 00:38:18,880
walk so another thing that's kind of

1043
00:38:23,750 --> 00:38:21,359
interesting about this is that i find

1044
00:38:26,150 --> 00:38:23,760
moving around in an environment

1045
00:38:28,069 --> 00:38:26,160
wearing a head-mounted display

1046
00:38:30,950 --> 00:38:28,079
using a controller like this fairly

1047
00:38:34,310 --> 00:38:30,960
disorienting and the reason why is that

1048
00:38:36,550 --> 00:38:34,320
the motion of my virtual body is not in

1049
00:38:38,069 --> 00:38:36,560
sync with what i'm seeing on the goggles

1050
00:38:39,670 --> 00:38:38,079
anymore

1051
00:38:41,589 --> 00:38:39,680
so remember that kinesthetic sense i was

1052
00:38:43,270 --> 00:38:41,599
talking about when the neck and what you

1053
00:38:45,190 --> 00:38:43,280
see in your eyes is in sync well all of

1054
00:38:46,790 --> 00:38:45,200
a sudden now what i'm seeing in my eyes

1055
00:38:48,390 --> 00:38:46,800
is out of sync with my body because my

1056
00:38:50,550 --> 00:38:48,400
kinesthetic sense says you are standing

1057
00:38:52,630 --> 00:38:50,560
still your legs are not moving

1058
00:38:54,150 --> 00:38:52,640
but the scene is moving this is why

1059
00:38:55,670 --> 00:38:54,160
sometimes people get motion sick in a

1060
00:38:57,510 --> 00:38:55,680
situation like this because actually

1061
00:38:58,790 --> 00:38:57,520
what your brain is saying is this does

1062
00:39:00,470 --> 00:38:58,800
not make sense

1063
00:39:02,390 --> 00:39:00,480
and i think you may have ingested some

1064
00:39:04,550 --> 00:39:02,400
kind of poison which is making you

1065
00:39:05,990 --> 00:39:04,560
hallucinogenic and so maybe it should

1066
00:39:06,870 --> 00:39:06,000
all come out of your stomach now which

1067
00:39:08,710 --> 00:39:06,880
is why

1068
00:39:09,589 --> 00:39:08,720
you start to feel sick to your stomach

1069
00:39:12,150 --> 00:39:09,599
so

1070
00:39:19,910 --> 00:39:12,160
we don't want that to happen of course

1071
00:39:24,069 --> 00:39:21,829
well we could allow ourselves to move

1072
00:39:25,670 --> 00:39:24,079
around by walking

1073
00:39:28,710 --> 00:39:25,680
this is an omnidirectional treadmill

1074
00:39:31,109 --> 00:39:28,720
called the virtuex omni and was in our

1075
00:39:33,670 --> 00:39:31,119
lab recently and that's brian duran one

1076
00:39:35,030 --> 00:39:33,680
of the members of our lab uh walking

1077
00:39:36,150 --> 00:39:35,040
around in that same scene that i was

1078
00:39:37,910 --> 00:39:36,160
just in

1079
00:39:40,550 --> 00:39:37,920
so the omni is kind of interesting it

1080
00:39:43,589 --> 00:39:40,560
accomplishes this feat by really being a

1081
00:39:45,670 --> 00:39:43,599
carefully engineered slippery bowl

1082
00:39:47,589 --> 00:39:45,680
it's really that

1083
00:39:49,670 --> 00:39:47,599
so you wear special shoes and when you

1084
00:39:51,109 --> 00:39:49,680
step outside the center you kind of

1085
00:39:53,510 --> 00:39:51,119
slide back towards the middle i'll show

1086
00:39:54,390 --> 00:39:53,520
it to you here again in a second

1087
00:39:56,150 --> 00:39:54,400
so

1088
00:39:57,270 --> 00:39:56,160
there's nothing no motorized no motors

1089
00:39:58,310 --> 00:39:57,280
or anything down there you're just

1090
00:40:00,310 --> 00:39:58,320
sliding back to the center and you're

1091
00:40:02,710 --> 00:40:00,320
balancing yourself a little bit on that

1092
00:40:04,069 --> 00:40:02,720
that waistband thing to uh because it's

1093
00:40:05,750 --> 00:40:04,079
sort of slippery

1094
00:40:08,150 --> 00:40:05,760
you can also wear a belt that will you

1095
00:40:11,430 --> 00:40:08,160
know keep you uh supported on that ring

1096
00:40:14,390 --> 00:40:11,440
that's around your waist

1097
00:40:16,069 --> 00:40:14,400
if you use a system like this one

1098
00:40:17,670 --> 00:40:16,079
i think that you would find

1099
00:40:20,150 --> 00:40:17,680
that that effect i was talking about

1100
00:40:22,790 --> 00:40:20,160
about moving around with a joystick is

1101
00:40:27,030 --> 00:40:22,800
less because now the scene moves when

1102
00:40:29,270 --> 00:40:27,040
your body moves so as you walk

1103
00:40:31,030 --> 00:40:29,280
the scene moves as if you're walking and

1104
00:40:33,270 --> 00:40:31,040
your body feels your legs moving and

1105
00:40:34,950 --> 00:40:33,280
sees the scene moving so less of that

1106
00:40:37,750 --> 00:40:34,960
discontinuity you know where your brain

1107
00:40:39,109 --> 00:40:37,760
says this doesn't make sense

1108
00:40:40,870 --> 00:40:39,119
we still don't have the position of the

1109
00:40:42,470 --> 00:40:40,880
head though right turns out that the

1110
00:40:45,030 --> 00:40:42,480
position of the head is really really

1111
00:40:46,310 --> 00:40:45,040
important you're using it right now you

1112
00:40:47,750 --> 00:40:46,320
may not really realize it but you're

1113
00:40:49,829 --> 00:40:47,760
pretty much spending your whole life

1114
00:40:51,589 --> 00:40:49,839
going like this

1115
00:40:53,910 --> 00:40:51,599
and by doing this you can start to get a

1116
00:40:55,829 --> 00:40:53,920
feeling for how far things are away how

1117
00:40:57,109 --> 00:40:55,839
much room there is you know could you

1118
00:40:58,470 --> 00:40:57,119
fit through there could you could you

1119
00:41:00,390 --> 00:40:58,480
make it out that door or something like

1120
00:41:03,190 --> 00:41:00,400
that

1121
00:41:04,710 --> 00:41:03,200
so here's what we can do to start also

1122
00:41:07,270 --> 00:41:04,720
tracking the head this is victor louelle

1123
00:41:09,190 --> 00:41:07,280
another member of our lab

1124
00:41:10,390 --> 00:41:09,200
he's got the oculus rift on again but

1125
00:41:13,190 --> 00:41:10,400
we're tracking it with these little

1126
00:41:15,589 --> 00:41:13,200
markers which are observed by these

1127
00:41:17,109 --> 00:41:15,599
cameras it's a vikon bonita system

1128
00:41:18,630 --> 00:41:17,119
same sort of like motion capture system

1129
00:41:19,910 --> 00:41:18,640
that they use in movies for character

1130
00:41:22,069 --> 00:41:19,920
animation

1131
00:41:23,510 --> 00:41:22,079
so using that we can not only keep track

1132
00:41:26,790 --> 00:41:23,520
of which way

1133
00:41:29,430 --> 00:41:26,800
the display is facing but also where the

1134
00:41:30,950 --> 00:41:29,440
display is within that space so we've

1135
00:41:32,630 --> 00:41:30,960
also dialed up the detail quite a lot

1136
00:41:34,230 --> 00:41:32,640
here we're also including the surface

1137
00:41:36,390 --> 00:41:34,240
detail in addition to orbital

1138
00:41:38,150 --> 00:41:36,400
information all blended in with pictures

1139
00:41:41,190 --> 00:41:38,160
taken by the robot as well

1140
00:41:43,349 --> 00:41:41,200
so when victor stands up looks around

1141
00:41:45,030 --> 00:41:43,359
when he starts walking in this

1142
00:41:46,550 --> 00:41:45,040
environment as he's going to do right

1143
00:41:49,190 --> 00:41:46,560
here

1144
00:41:51,910 --> 00:41:49,200
the scene changes as if he's walking

1145
00:41:54,230 --> 00:41:51,920
around in that same spot on mars he can

1146
00:41:55,349 --> 00:41:54,240
crouch down and take a new perspective

1147
00:41:57,190 --> 00:41:55,359
on the scene

1148
00:41:59,349 --> 00:41:57,200
study a wheel

1149
00:42:00,870 --> 00:41:59,359
he can use that head parallax that i was

1150
00:42:02,870 --> 00:42:00,880
talking about moving your head back and

1151
00:42:04,230 --> 00:42:02,880
forth to get a feeling for how are those

1152
00:42:06,550 --> 00:42:04,240
things really situated in the

1153
00:42:08,309 --> 00:42:06,560
environment how big are they how much

1154
00:42:10,470 --> 00:42:08,319
room is there in between them you can

1155
00:42:12,390 --> 00:42:10,480
really start to sort of feel how large

1156
00:42:15,430 --> 00:42:12,400
things are in the environment that he's

1157
00:42:17,589 --> 00:42:15,440
been asked to explore

1158
00:42:19,349 --> 00:42:17,599
so let's review those three ingredients

1159
00:42:21,190 --> 00:42:19,359
we put together there

1160
00:42:23,190 --> 00:42:21,200
we connected what you see with the

1161
00:42:24,710 --> 00:42:23,200
direction you're facing with where your

1162
00:42:26,790 --> 00:42:24,720
neck is pointing

1163
00:42:28,150 --> 00:42:26,800
we made it possible for you to move by

1164
00:42:30,309 --> 00:42:28,160
walking

1165
00:42:33,030 --> 00:42:30,319
and we kept track of your point of view

1166
00:42:35,510 --> 00:42:33,040
by tracking the position of your head

1167
00:42:37,589 --> 00:42:35,520
those three ingredients turn out to be a

1168
00:42:39,109 --> 00:42:37,599
really really great combination

1169
00:42:40,950 --> 00:42:39,119
and then although the system is still

1170
00:42:42,150 --> 00:42:40,960
early and we still have more research to

1171
00:42:43,670 --> 00:42:42,160
do and we have some studies that we're

1172
00:42:45,829 --> 00:42:43,680
going to be doing testing the the

1173
00:42:47,829 --> 00:42:45,839
efficacy of this comparing it with other

1174
00:42:49,670 --> 00:42:47,839
means of trying to understand

1175
00:42:51,270 --> 00:42:49,680
an environment that's being explored

1176
00:42:52,870 --> 00:42:51,280
we're very encouraged by the results

1177
00:42:55,430 --> 00:42:52,880
that we're seeing so far

1178
00:43:01,030 --> 00:42:55,440
i'm really looking forward to sharing

1179
00:43:05,270 --> 00:43:03,030
i wish i could bring all of you up here

1180
00:43:09,829 --> 00:43:05,280
right now

1181
00:43:10,829 --> 00:43:09,839
and believe me

1182
00:43:13,510 --> 00:43:10,839
in my

1183
00:43:15,910 --> 00:43:13,520
hopes what we're on is a path towards

1184
00:43:17,990 --> 00:43:15,920
where these kinds of technologies are

1185
00:43:19,750 --> 00:43:18,000
going to make it possible for all of us

1186
00:43:21,750 --> 00:43:19,760
to experience that but i wanted to give

1187
00:43:24,309 --> 00:43:21,760
you a little bit of taste of it tonight

1188
00:43:25,349 --> 00:43:24,319
so i love doing live demos in my talks

1189
00:43:26,790 --> 00:43:25,359
and if i'm going to try to do something

1190
00:43:28,069 --> 00:43:26,800
even a little crazier here which is that

1191
00:43:30,230 --> 00:43:28,079
i'm going to try to do a demo that

1192
00:43:31,670 --> 00:43:30,240
involves all of you right now and

1193
00:43:32,710 --> 00:43:31,680
everybody out there who's watching on

1194
00:43:34,470 --> 00:43:32,720
online

1195
00:43:36,069 --> 00:43:34,480
so if you'll pull out your phone right

1196
00:43:38,550 --> 00:43:36,079
now yes i'm actually telling you at a

1197
00:43:40,230 --> 00:43:38,560
talk to pull out your phone which i know

1198
00:43:43,510 --> 00:43:40,240
is a little bit rare a little bit out of

1199
00:43:46,790 --> 00:43:45,349
type in that url that's at the top into

1200
00:43:48,630 --> 00:43:46,800
the browser and i should warn you that

1201
00:43:49,990 --> 00:43:48,640
if you have an iphone 4 or 5 this is

1202
00:43:51,990 --> 00:43:50,000
going to work

1203
00:43:54,309 --> 00:43:52,000
really well if you have an android phone

1204
00:43:55,910 --> 00:43:54,319
it may not work quite as well turns out

1205
00:43:58,870 --> 00:43:55,920
that different phones have a different

1206
00:44:00,790 --> 00:43:58,880
amount of noise in the gyroscope which

1207
00:44:03,030 --> 00:44:00,800
is keeping track of which way the phone

1208
00:44:04,710 --> 00:44:03,040
is facing so if it starts to get really

1209
00:44:05,750 --> 00:44:04,720
jittery and shaky on you first of all

1210
00:44:07,190 --> 00:44:05,760
make sure that you've turned your phone

1211
00:44:08,550 --> 00:44:07,200
this way and that you're holding it sort

1212
00:44:10,950 --> 00:44:08,560
of like you would if you were taking a

1213
00:44:13,430 --> 00:44:10,960
picture by the way stand up

1214
00:44:15,190 --> 00:44:13,440
because it's a lot easier when you can

1215
00:44:17,910 --> 00:44:15,200
look around behind you

1216
00:44:19,109 --> 00:44:17,920
look around this environment

1217
00:44:20,829 --> 00:44:19,119
if you're having troubles with your

1218
00:44:30,870 --> 00:44:20,839
phone borrow your

1219
00:44:34,069 --> 00:44:32,150
and then move your phone around as if

1220
00:44:35,349 --> 00:44:34,079
you were sort of taking a picture and as

1221
00:44:36,710 --> 00:44:35,359
you turn around it's going to change

1222
00:44:38,870 --> 00:44:36,720
your perspective and it should give you

1223
00:44:40,230 --> 00:44:38,880
just a little taste of what i'm talking

1224
00:44:42,470 --> 00:44:40,240
about here

1225
00:44:44,390 --> 00:44:42,480
it's the same panorama that i challenged

1226
00:44:46,069 --> 00:44:44,400
you with those questions about early on

1227
00:44:47,589 --> 00:44:46,079
in the presentation the one where i was

1228
00:44:49,109 --> 00:44:47,599
asking you which way you would walk it's

1229
00:44:51,430 --> 00:44:49,119
the same panorama shown down here at the

1230
00:44:54,230 --> 00:44:51,440
bottom if you look down you'll see the

1231
00:44:55,750 --> 00:44:54,240
rover underneath

1232
00:44:57,349 --> 00:44:55,760
if you look

1233
00:44:59,670 --> 00:44:57,359
try to find the rover tracks in the

1234
00:45:01,829 --> 00:44:59,680
environment

1235
00:45:08,150 --> 00:45:01,839
you can look and see the hills out in

1236
00:45:13,349 --> 00:45:09,510
and then i want you

1237
00:45:14,870 --> 00:45:13,359
to try and mentally extend

1238
00:45:16,470 --> 00:45:14,880
how this feels

1239
00:45:17,829 --> 00:45:16,480
with how it would feel with even more

1240
00:45:19,750 --> 00:45:17,839
technology because really this is a

1241
00:45:21,990 --> 00:45:19,760
cheap little trick this is a tiny little

1242
00:45:25,030 --> 00:45:22,000
technological trick

1243
00:45:27,589 --> 00:45:25,040
imagine what where is possible with more

1244
00:45:31,990 --> 00:45:27,599
technology with more efforts put into

1245
00:45:35,829 --> 00:45:33,430
so here's what gets me kind of excited

1246
00:45:37,270 --> 00:45:35,839
about this because i would believe that

1247
00:45:39,510 --> 00:45:37,280
right now

1248
00:45:41,670 --> 00:45:39,520
all of us in this room

1249
00:45:44,150 --> 00:45:41,680
and the people who are watching online

1250
00:45:48,150 --> 00:45:44,160
are standing on mars together and we're

1251
00:45:52,710 --> 00:45:50,790
and it's my hope

1252
00:45:54,870 --> 00:45:52,720
that when we come to a day like the day

1253
00:45:56,550 --> 00:45:54,880
when we first put an astronaut's boot on

1254
00:45:58,470 --> 00:45:56,560
that planet

1255
00:46:01,589 --> 00:45:58,480
that they're going to be a billion

1256
00:46:03,349 --> 00:46:01,599
people in attendance for that event

1257
00:46:04,230 --> 00:46:03,359
that they're going to be standing beside

1258
00:46:06,230 --> 00:46:04,240
them

1259
00:46:08,230 --> 00:46:06,240
standing beside each other and alongside

1260
00:46:10,390 --> 00:46:08,240
that astronaut on mars

1261
00:46:12,230 --> 00:46:10,400
i hope to see you all there and i thank

1262
00:46:13,109 --> 00:46:12,240
you very much for your attention this

1263
00:46:24,870 --> 00:46:13,119
evening

1264
00:46:27,829 --> 00:46:26,309
and i've left time for questions so i'm

1265
00:46:29,190 --> 00:46:27,839
happy to take them i'm supposed to

1266
00:46:30,950 --> 00:46:29,200
remind you that if you have a question

1267
00:46:32,470 --> 00:46:30,960
to please come to this microphone here

1268
00:46:34,790 --> 00:46:32,480
so that everybody can hear the question

1269
00:46:38,309 --> 00:46:34,800
as well as the people who are watching

1270
00:46:46,550 --> 00:46:39,430
and i'll leave this running for a while

1271
00:46:52,470 --> 00:46:49,510
uh at the beginning you showed the

1272
00:46:53,750 --> 00:46:52,480
projection of the

1273
00:46:57,109 --> 00:46:53,760
model

1274
00:47:00,150 --> 00:46:57,119
vehicle ahead used in the uncertainty

1275
00:47:04,390 --> 00:47:02,230
how do you how accurately can you

1276
00:47:07,670 --> 00:47:04,400
actually calculate where the boundaries

1277
00:47:09,589 --> 00:47:07,680
of that uncertainty cloud are and how do

1278
00:47:12,069 --> 00:47:09,599
you do that so it's challenging of

1279
00:47:12,950 --> 00:47:12,079
course uh we make models

1280
00:47:13,750 --> 00:47:12,960
of

1281
00:47:18,950 --> 00:47:13,760
the

1282
00:47:20,710 --> 00:47:18,960
terrain that we're driving in and

1283
00:47:21,430 --> 00:47:20,720
those models are actually they're

1284
00:47:24,710 --> 00:47:21,440
they're

1285
00:47:28,069 --> 00:47:24,720
that challenge of simulating and

1286
00:47:31,030 --> 00:47:28,079
modeling the interactions of robots with

1287
00:47:33,670 --> 00:47:31,040
surfaces with so wheels on dirt and how

1288
00:47:36,230 --> 00:47:33,680
they react so the answer is we do our

1289
00:47:38,790 --> 00:47:36,240
best and that uncertainty

1290
00:47:41,030 --> 00:47:38,800
volume is really an expression of

1291
00:47:43,990 --> 00:47:41,040
how poorly really we're doing meaning

1292
00:47:46,150 --> 00:47:44,000
that the larger that volume is the

1293
00:47:47,430 --> 00:47:46,160
the more error there is in our guesses

1294
00:47:49,190 --> 00:47:47,440
about where the robot's going to be if

1295
00:47:50,550 --> 00:47:49,200
we could predict perfectly there would

1296
00:47:52,230 --> 00:47:50,560
be no uncertainty volume there would be

1297
00:47:53,109 --> 00:47:52,240
just a point the robot will be exactly

1298
00:47:56,309 --> 00:47:53,119
here

1299
00:47:58,069 --> 00:47:56,319
so uh i think the answer would be um i

1300
00:48:00,230 --> 00:47:58,079
think we do it adequately well to make

1301
00:48:02,470 --> 00:48:00,240
this mode of operations reasonable but

1302
00:48:03,430 --> 00:48:02,480
there is much more room for improvement

1303
00:48:06,150 --> 00:48:03,440
well like

1304
00:48:08,309 --> 00:48:06,160
for example would like if it if it uh

1305
00:48:09,910 --> 00:48:08,319
the wheel hit a slipped on a rock or

1306
00:48:11,510 --> 00:48:09,920
something would that

1307
00:48:13,030 --> 00:48:11,520
add to the uncertainty something like

1308
00:48:14,790 --> 00:48:13,040
that absolutely yeah it's a wheel

1309
00:48:16,390 --> 00:48:14,800
slipping in soft soil

1310
00:48:17,109 --> 00:48:16,400
falling off of a rock things like that

1311
00:48:19,670 --> 00:48:17,119
now

1312
00:48:21,270 --> 00:48:19,680
we have measures on the mars rovers for

1313
00:48:22,790 --> 00:48:21,280
instance that we can use to counteract

1314
00:48:24,950 --> 00:48:22,800
that one of them is called visual

1315
00:48:27,829 --> 00:48:24,960
odometry so when we have this

1316
00:48:30,150 --> 00:48:27,839
capability running on the robots

1317
00:48:31,750 --> 00:48:30,160
they can keep track of

1318
00:48:33,750 --> 00:48:31,760
how things are moving past them in the

1319
00:48:35,670 --> 00:48:33,760
environment visually so the the

1320
00:48:38,790 --> 00:48:35,680
challenges is that if you're only using

1321
00:48:40,870 --> 00:48:38,800
the rover's uh inertial measurement and

1322
00:48:42,710 --> 00:48:40,880
it's you know the turning of its wheels

1323
00:48:44,069 --> 00:48:42,720
there's a lot of drift is what we would

1324
00:48:45,750 --> 00:48:44,079
call there's a lot of error that builds

1325
00:48:47,910 --> 00:48:45,760
up over time something like visual

1326
00:48:50,069 --> 00:48:47,920
odometry reduces but doesn't eliminate

1327
00:48:52,069 --> 00:48:50,079
that error and also visual odometry at

1328
00:48:53,829 --> 00:48:52,079
times can be expensive to run it's a

1329
00:48:55,670 --> 00:48:53,839
computationally intensive task on the

1330
00:48:57,829 --> 00:48:55,680
robot so we don't always use it we

1331
00:48:58,950 --> 00:48:57,839
usually use it when we expect slip to be

1332
00:49:00,829 --> 00:48:58,960
very high

1333
00:49:03,270 --> 00:49:00,839
thank you

1334
00:49:07,349 --> 00:49:03,280
sure uh hello

1335
00:49:08,630 --> 00:49:07,359
uh was the dispute resolved between

1336
00:49:10,150 --> 00:49:08,640
uh jean

1337
00:49:12,309 --> 00:49:10,160
doggoff and

1338
00:49:14,470 --> 00:49:12,319
gene rotterdam

1339
00:49:17,670 --> 00:49:14,480
regarding the

1340
00:49:18,710 --> 00:49:17,680
control panel in the starship enterprise

1341
00:49:21,750 --> 00:49:18,720
when

1342
00:49:22,950 --> 00:49:21,760
rotterdam said well he couldn't really

1343
00:49:25,910 --> 00:49:22,960
get his

1344
00:49:27,990 --> 00:49:25,920
mind and the human brain around this

1345
00:49:30,150 --> 00:49:28,000
immersion

1346
00:49:32,950 --> 00:49:30,160
i really wish i could speak on that with

1347
00:49:35,670 --> 00:49:32,960
authority i you know although i uh you

1348
00:49:37,910 --> 00:49:35,680
know i'm a fan of the show i i can't say

1349
00:49:39,990 --> 00:49:37,920
that i remember that specific episode or

1350
00:49:45,270 --> 00:49:40,000
that specific question

1351
00:49:48,950 --> 00:49:47,030
um could you clarify a little bit on how

1352
00:49:50,790 --> 00:49:48,960
you plan on uh incorporating the two

1353
00:49:52,309 --> 00:49:50,800
technologies with like the movement

1354
00:49:53,270 --> 00:49:52,319
prediction and also the oculus rift

1355
00:49:55,430 --> 00:49:53,280
stuff

1356
00:49:56,230 --> 00:49:55,440
sure yeah so

1357
00:49:59,670 --> 00:49:56,240
um

1358
00:50:00,870 --> 00:49:59,680
when you consider that

1359
00:50:02,710 --> 00:50:00,880
observing the environment and

1360
00:50:03,829 --> 00:50:02,720
understanding is really understanding

1361
00:50:05,349 --> 00:50:03,839
the environment is really just sort of

1362
00:50:07,829 --> 00:50:05,359
part one of the challenge we also need

1363
00:50:09,270 --> 00:50:07,839
to affect things in that environment uh

1364
00:50:11,030 --> 00:50:09,280
you can imagine that the second half of

1365
00:50:12,630 --> 00:50:11,040
the talk is sort of the observation and

1366
00:50:14,470 --> 00:50:12,640
understanding part and then the first

1367
00:50:17,510 --> 00:50:14,480
part of the talk is then how you then

1368
00:50:19,109 --> 00:50:17,520
begin to move in effect so you could

1369
00:50:21,430 --> 00:50:19,119
imagine

1370
00:50:23,750 --> 00:50:21,440
you know donning a system like this or

1371
00:50:25,829 --> 00:50:23,760
standing inside a you know a theater a

1372
00:50:28,470 --> 00:50:25,839
holodeck like theater planning where you

1373
00:50:30,150 --> 00:50:28,480
want to go and then using the time delay

1374
00:50:32,230 --> 00:50:30,160
mitigation strategies that i discussed

1375
00:50:33,910 --> 00:50:32,240
at the beginning to improve the

1376
00:50:36,390 --> 00:50:33,920
efficiency of what you're accomplishing

1377
00:50:38,950 --> 00:50:36,400
there so for instance being able to

1378
00:50:40,710 --> 00:50:38,960
direct it where to drive and then stay

1379
00:50:43,829 --> 00:50:40,720
ahead of it such that you're not wasting

1380
00:50:45,589 --> 00:50:43,839
a lot of time uh on mars i see would you

1381
00:50:47,109 --> 00:50:45,599
simulate like or

1382
00:50:48,309 --> 00:50:47,119
like a fake environment for the person

1383
00:50:49,829 --> 00:50:48,319
to see and then observe and then make

1384
00:50:51,190 --> 00:50:49,839
decisions that way

1385
00:50:52,390 --> 00:50:51,200
well you can fake something so for

1386
00:50:54,470 --> 00:50:52,400
instance like i showed with pitfall

1387
00:50:56,390 --> 00:50:54,480
harry right you and also with the

1388
00:50:58,230 --> 00:50:56,400
committed state of the robot you would

1389
00:50:59,750 --> 00:50:58,240
certainly want to show them your best

1390
00:51:01,750 --> 00:50:59,760
guess about where the robot's going to

1391
00:51:03,750 --> 00:51:01,760
end up if you send those commands and i

1392
00:51:06,390 --> 00:51:03,760
think also it's a value to show them

1393
00:51:07,670 --> 00:51:06,400
that extra state the committed state of

1394
00:51:09,030 --> 00:51:07,680
where it would be if you stopped right

1395
00:51:10,470 --> 00:51:09,040
this moment

1396
00:51:12,790 --> 00:51:10,480
so you certainly want to fake those

1397
00:51:14,390 --> 00:51:12,800
things

1398
00:51:16,470 --> 00:51:14,400
i think that there's a trade-off when

1399
00:51:18,710 --> 00:51:16,480
you fake more that basically you're

1400
00:51:20,390 --> 00:51:18,720
trading the

1401
00:51:21,589 --> 00:51:20,400
efficiencies that you gain from being

1402
00:51:23,910 --> 00:51:21,599
able to get a little glimpse of what

1403
00:51:25,829 --> 00:51:23,920
might be coming with the risks that

1404
00:51:27,510 --> 00:51:25,839
you're taking if you're wrong and so i

1405
00:51:29,270 --> 00:51:27,520
would say you should predict as much as

1406
00:51:31,349 --> 00:51:29,280
you possibly can as long as you can

1407
00:51:32,549 --> 00:51:31,359
simultaneously communicate the

1408
00:51:35,109 --> 00:51:32,559
uncertainty you have in those

1409
00:51:38,790 --> 00:51:35,119
predictions so that people don't start

1410
00:51:42,230 --> 00:51:38,800
believing something that was only a gas

1411
00:51:45,910 --> 00:51:44,230
so the the fully immersive stuff is cool

1412
00:51:46,870 --> 00:51:45,920
but up into that point it seems like

1413
00:51:48,630 --> 00:51:46,880
there's a

1414
00:51:51,109 --> 00:51:48,640
first step which is just generating that

1415
00:51:53,030 --> 00:51:51,119
sort of full 3d environment a prediction

1416
00:51:53,990 --> 00:51:53,040
of it about the rover how close are we

1417
00:51:55,190 --> 00:51:54,000
to being able to do that for mission

1418
00:51:56,549 --> 00:51:55,200
planning never mind the head mounted

1419
00:51:57,750 --> 00:51:56,559
displays but just

1420
00:51:59,829 --> 00:51:57,760
you know the

1421
00:52:02,309 --> 00:51:59,839
first person shooter style controls so

1422
00:52:04,710 --> 00:52:02,319
the 3d engineering environment yeah

1423
00:52:05,990 --> 00:52:04,720
okay uh

1424
00:52:08,470 --> 00:52:06,000
well first of all let's talk about the

1425
00:52:10,870 --> 00:52:08,480
sensors that we have on our robots to

1426
00:52:14,390 --> 00:52:10,880
feed such an environment so you know the

1427
00:52:16,470 --> 00:52:14,400
mars rovers for example have on them

1428
00:52:18,309 --> 00:52:16,480
a set of cameras on the mast which

1429
00:52:20,870 --> 00:52:18,319
points in different directions and it's

1430
00:52:22,870 --> 00:52:20,880
those images that we're using to render

1431
00:52:25,030 --> 00:52:22,880
the environment that you saw uh the one

1432
00:52:27,349 --> 00:52:25,040
where we were tracking the head position

1433
00:52:28,710 --> 00:52:27,359
uh the so the navigation camera pictures

1434
00:52:29,990 --> 00:52:28,720
were actually what we were showing there

1435
00:52:31,910 --> 00:52:30,000
we're not even actually showing the

1436
00:52:33,910 --> 00:52:31,920
highest resolution data that curiosity

1437
00:52:35,510 --> 00:52:33,920
can acquire which is with the mast cam

1438
00:52:38,069 --> 00:52:35,520
we were just using the navigation images

1439
00:52:40,950 --> 00:52:38,079
which are good but lower resolution so i

1440
00:52:44,630 --> 00:52:40,960
think we are in a good place there we

1441
00:52:46,309 --> 00:52:44,640
um there are whole parts of jpl who have

1442
00:52:48,069 --> 00:52:46,319
really i think mastered the technique of

1443
00:52:49,910 --> 00:52:48,079
taking images

1444
00:52:51,829 --> 00:52:49,920
that are acquired in stereo so two

1445
00:52:54,549 --> 00:52:51,839
images and extracting ranged data from

1446
00:52:56,069 --> 00:52:54,559
it and then converting that into a

1447
00:52:58,870 --> 00:52:56,079
terrain model like the one that you were

1448
00:53:00,630 --> 00:52:58,880
seeing there now one challenge is then

1449
00:53:02,150 --> 00:53:00,640
effective rendering of that so i'll go

1450
00:53:03,349 --> 00:53:02,160
just a little bit more into depth about

1451
00:53:04,470 --> 00:53:03,359
some of the tricks we've been playing

1452
00:53:05,589 --> 00:53:04,480
there we had

1453
00:53:07,270 --> 00:53:05,599
actually a student who worked with us

1454
00:53:08,790 --> 00:53:07,280
this summer charlie goddard who did some

1455
00:53:11,190 --> 00:53:08,800
work in this area

1456
00:53:14,710 --> 00:53:11,200
and we're actually blending and i'll go

1457
00:53:27,030 --> 00:53:14,720
back to that video so you can see

1458
00:53:34,390 --> 00:53:29,670
okay so

1459
00:53:39,109 --> 00:53:37,430
okay um that's a little hard to see but

1460
00:53:41,990 --> 00:53:39,119
there's actually a couple kinds of data

1461
00:53:45,270 --> 00:53:42,000
being shown here up near the top are

1462
00:53:46,710 --> 00:53:45,280
two-dimensional images that are being so

1463
00:53:48,390 --> 00:53:46,720
when the rover takes its pictures of

1464
00:53:50,790 --> 00:53:48,400
course it's just taking pictures you

1465
00:53:53,030 --> 00:53:50,800
know like photographs

1466
00:53:55,510 --> 00:53:53,040
we try to use those pictures as long as

1467
00:53:57,670 --> 00:53:55,520
we can because the date the detail that

1468
00:53:58,950 --> 00:53:57,680
a picture offers is very very high as

1469
00:54:00,630 --> 00:53:58,960
long as you're looking at the scene from

1470
00:54:02,230 --> 00:54:00,640
the same point of view as the picture

1471
00:54:04,230 --> 00:54:02,240
you can get a great deal of information

1472
00:54:06,069 --> 00:54:04,240
from from the photograph the problem is

1473
00:54:08,069 --> 00:54:06,079
when your head moves away from where

1474
00:54:09,990 --> 00:54:08,079
that picture was taken because that's

1475
00:54:11,750 --> 00:54:10,000
when the image no longer really makes

1476
00:54:14,150 --> 00:54:11,760
sense now one thing that charlie worked

1477
00:54:16,710 --> 00:54:14,160
on this summer was a way to use a

1478
00:54:19,670 --> 00:54:16,720
technique we were calling

1479
00:54:21,430 --> 00:54:19,680
disparity mapping to take pixels from

1480
00:54:23,190 --> 00:54:21,440
that image projected through the model

1481
00:54:26,710 --> 00:54:23,200
of the camera that we have

1482
00:54:28,470 --> 00:54:26,720
and shift them as your head moves and so

1483
00:54:30,069 --> 00:54:28,480
what that allows us to do is to hang on

1484
00:54:32,309 --> 00:54:30,079
to those images just a little bit longer

1485
00:54:34,630 --> 00:54:32,319
so if i move my head to the side we can

1486
00:54:36,230 --> 00:54:34,640
still show basically the right thing on

1487
00:54:38,150 --> 00:54:36,240
that image and if we go a little bit

1488
00:54:40,230 --> 00:54:38,160
further we have to give up on the image

1489
00:54:42,870 --> 00:54:40,240
and we give up in favor of what you see

1490
00:54:45,109 --> 00:54:42,880
in the foreground of this video and that

1491
00:54:47,190 --> 00:54:45,119
is all point cloud data so point cloud

1492
00:54:49,510 --> 00:54:47,200
data is then taking those images which

1493
00:54:51,349 --> 00:54:49,520
give us information about how far away

1494
00:54:52,470 --> 00:54:51,359
things are in the environment and

1495
00:54:54,390 --> 00:54:52,480
rendering

1496
00:54:57,109 --> 00:54:54,400
a mesh you can think of it as a bunch of

1497
00:54:58,950 --> 00:54:57,119
dots connected together

1498
00:55:00,630 --> 00:54:58,960
that give you the the feeling that it's

1499
00:55:02,069 --> 00:55:00,640
a continuous scene we're actually using

1500
00:55:03,750 --> 00:55:02,079
a technique here called surflit

1501
00:55:06,630 --> 00:55:03,760
rendering which another one of our

1502
00:55:07,349 --> 00:55:06,640
students dylan worked on this summer

1503
00:55:09,109 --> 00:55:07,359
and

1504
00:55:10,470 --> 00:55:09,119
the

1505
00:55:11,750 --> 00:55:10,480
interesting thing about surflit

1506
00:55:13,750 --> 00:55:11,760
rendering is that it's a little more

1507
00:55:15,109 --> 00:55:13,760
data efficient than other methods that

1508
00:55:17,030 --> 00:55:15,119
have been used so what we're looking for

1509
00:55:19,270 --> 00:55:17,040
actually is a great combination of all

1510
00:55:20,390 --> 00:55:19,280
these rendering techniques all used at

1511
00:55:23,190 --> 00:55:20,400
the same time

1512
00:55:25,589 --> 00:55:23,200
to give you the most complete scene that

1513
00:55:27,990 --> 00:55:25,599
uh that we can so i would say this is

1514
00:55:29,589 --> 00:55:28,000
close to is as good as we can do it

1515
00:55:30,470 --> 00:55:29,599
right now we hope to get further than

1516
00:55:32,150 --> 00:55:30,480
this

1517
00:55:33,910 --> 00:55:32,160
the next opportunity so the next

1518
00:55:35,190 --> 00:55:33,920
frontier would be thinking about new

1519
00:55:37,589 --> 00:55:35,200
instruments that could be put on a

1520
00:55:39,510 --> 00:55:37,599
future mission that are designed to

1521
00:55:42,230 --> 00:55:39,520
build environments like these

1522
00:55:43,750 --> 00:55:42,240
so not piecing together from images

1523
00:55:47,270 --> 00:55:43,760
necessarily but what other kinds of

1524
00:55:49,109 --> 00:55:47,280
instruments lidar-based instruments

1525
00:55:50,309 --> 00:55:49,119
things like that that could build up 3d

1526
00:55:51,750 --> 00:55:50,319
scenes

1527
00:55:53,190 --> 00:55:51,760
is most of the image data you get from

1528
00:55:54,549 --> 00:55:53,200
curiosity usable for something like this

1529
00:55:56,230 --> 00:55:54,559
or is it something you can only do with

1530
00:55:57,829 --> 00:55:56,240
the sort of stereo panoramas that you

1531
00:55:59,430 --> 00:55:57,839
get here really only the stereo

1532
00:56:01,990 --> 00:55:59,440
panoramas because

1533
00:56:04,230 --> 00:56:02,000
we need the the stereo information to

1534
00:56:06,470 --> 00:56:04,240
help us understand um

1535
00:56:08,549 --> 00:56:06,480
how far away things are on the scene in

1536
00:56:11,349 --> 00:56:08,559
some cases uh monoscopic you know just

1537
00:56:13,109 --> 00:56:11,359
one eye image uh is is useful

1538
00:56:14,710 --> 00:56:13,119
another challenge is like right now like

1539
00:56:16,470 --> 00:56:14,720
we're only using one camera here we're

1540
00:56:19,270 --> 00:56:16,480
using well we're using a pair of cameras

1541
00:56:20,789 --> 00:56:19,280
but one type of camera on curiosity we

1542
00:56:23,270 --> 00:56:20,799
another interesting challenge is

1543
00:56:26,309 --> 00:56:23,280
integrating the other imaging sensors

1544
00:56:27,750 --> 00:56:26,319
like the mast cam or the hazard cameras

1545
00:56:29,109 --> 00:56:27,760
which are actually mounted on the body

1546
00:56:30,789 --> 00:56:29,119
of the vehicle

1547
00:56:31,670 --> 00:56:30,799
possibly even the camera that is on the

1548
00:56:35,190 --> 00:56:31,680
arm

1549
00:56:37,030 --> 00:56:35,200
the molle or the microscope basically

1550
00:56:38,309 --> 00:56:37,040
those all could produce useful data in

1551
00:56:39,510 --> 00:56:38,319
certain cases

1552
00:56:44,710 --> 00:56:39,520
that could help

1553
00:56:49,349 --> 00:56:47,510
hi um a couple of us were talking at

1554
00:56:50,630 --> 00:56:49,359
lunch about your talk we're excited to

1555
00:56:53,349 --> 00:56:50,640
come and uh

1556
00:56:54,870 --> 00:56:53,359
my friend asked me a question uh

1557
00:56:56,950 --> 00:56:54,880
you know when we're all standing in our

1558
00:56:58,710 --> 00:56:56,960
mars given the latency

1559
00:57:01,190 --> 00:56:58,720
how will we know what we're going to say

1560
00:57:02,870 --> 00:57:01,200
to one another or how we interact with

1561
00:57:04,470 --> 00:57:02,880
each other

1562
00:57:06,069 --> 00:57:04,480
we were wondering you know could you

1563
00:57:08,710 --> 00:57:06,079
build in some kind of predictive

1564
00:57:11,030 --> 00:57:08,720
modeling of of how i might react to you

1565
00:57:13,030 --> 00:57:11,040
and use some of the same techniques to

1566
00:57:15,270 --> 00:57:13,040
to convince this poor astronaut that may

1567
00:57:16,870 --> 00:57:15,280
have been out there for 500 days

1568
00:57:18,549 --> 00:57:16,880
or however long we're going to send him

1569
00:57:20,309 --> 00:57:18,559
out that maybe he's not alone in the

1570
00:57:21,190 --> 00:57:20,319
universe but that he has friends with

1571
00:57:23,349 --> 00:57:21,200
him

1572
00:57:25,750 --> 00:57:23,359
in a way that's a little bit more

1573
00:57:27,589 --> 00:57:25,760
instinctive than you know 25 minute

1574
00:57:28,789 --> 00:57:27,599
delay on how high are you doing

1575
00:57:30,230 --> 00:57:28,799
you know it's funny i don't think it's

1576
00:57:32,230 --> 00:57:30,240
so far fetched

1577
00:57:33,910 --> 00:57:32,240
so it kind of falls in that category of

1578
00:57:35,109 --> 00:57:33,920
of you know us all desperately wanting

1579
00:57:36,069 --> 00:57:35,119
to believe that we're more complicated

1580
00:57:37,430 --> 00:57:36,079
than we are

1581
00:57:39,430 --> 00:57:37,440
but uh if you look at for instance

1582
00:57:40,950 --> 00:57:39,440
something as simple as predictive

1583
00:57:42,390 --> 00:57:40,960
texting you know when you start typing

1584
00:57:43,910 --> 00:57:42,400
on your phone and it starts completing

1585
00:57:45,990 --> 00:57:43,920
the words for you

1586
00:57:48,150 --> 00:57:46,000
there is a lot of predictability it

1587
00:57:50,710 --> 00:57:48,160
turns out in in i think the way that

1588
00:57:51,750 --> 00:57:50,720
humans behave um i think there's a limit

1589
00:57:56,470 --> 00:57:51,760
you know we wouldn't be able to push

1590
00:57:58,069 --> 00:57:56,480
that too terribly far uh the

1591
00:57:59,430 --> 00:57:58,079
i think

1592
00:58:00,390 --> 00:57:59,440
predicting the directions that people

1593
00:58:02,069 --> 00:58:00,400
are going

1594
00:58:03,510 --> 00:58:02,079
the ways that they're going to interact

1595
00:58:05,990 --> 00:58:03,520
with objects in the environment things

1596
00:58:08,950 --> 00:58:06,000
like that seem like low-hanging fruit

1597
00:58:10,870 --> 00:58:08,960
conversations more difficult i hope i

1598
00:58:11,670 --> 00:58:10,880
hope i would not be so easily predicted

1599
00:58:13,190 --> 00:58:11,680
that

1600
00:58:15,109 --> 00:58:13,200
someone could uh could play out my

1601
00:58:17,510 --> 00:58:15,119
conversation before i said it only my

1602
00:58:18,309 --> 00:58:17,520
wife can do that

1603
00:58:20,829 --> 00:58:18,319
yes

1604
00:58:23,670 --> 00:58:20,839
um yeah in the beginning you showed like

1605
00:58:26,470 --> 00:58:23,680
um these different scenarios

1606
00:58:28,230 --> 00:58:26,480
with a like relay satellite in the in

1607
00:58:29,190 --> 00:58:28,240
the martian orbit

1608
00:58:31,589 --> 00:58:29,200
but

1609
00:58:33,910 --> 00:58:31,599
right now we don't have that one

1610
00:58:36,470 --> 00:58:33,920
can you anyhow already gain some

1611
00:58:38,230 --> 00:58:36,480
efficiencies or can you already use this

1612
00:58:40,150 --> 00:58:38,240
technology with the

1613
00:58:41,270 --> 00:58:40,160
with this 24-hour

1614
00:58:43,829 --> 00:58:41,280
uh

1615
00:58:46,150 --> 00:58:43,839
time frame communications

1616
00:58:48,789 --> 00:58:46,160
it's a good question um

1617
00:58:50,069 --> 00:58:48,799
i would say that the operations process

1618
00:58:51,910 --> 00:58:50,079
that we've developed for controlling

1619
00:58:54,630 --> 00:58:51,920
robots on mars

1620
00:58:56,069 --> 00:58:54,640
today is very well suited to the amount

1621
00:58:58,069 --> 00:58:56,079
of time delay that we're dealing with

1622
00:58:59,430 --> 00:58:58,079
and with that cadence of operations

1623
00:59:01,510 --> 00:58:59,440
where we're

1624
00:59:04,950 --> 00:59:01,520
you know uplinking really at most once a

1625
00:59:07,270 --> 00:59:04,960
day often at times once every three days

1626
00:59:08,309 --> 00:59:07,280
in certain cases

1627
00:59:10,069 --> 00:59:08,319
uh

1628
00:59:12,630 --> 00:59:10,079
we do have orbiters around mars of

1629
00:59:14,870 --> 00:59:12,640
course and they are relaying almost all

1630
00:59:17,030 --> 00:59:14,880
of the data that comes back from from

1631
00:59:19,270 --> 00:59:17,040
the assets that are on the surface it's

1632
00:59:21,030 --> 00:59:19,280
that there are too few and they have

1633
00:59:22,789 --> 00:59:21,040
other jobs you know these these orbiters

1634
00:59:24,470 --> 00:59:22,799
are are still performing scientific

1635
00:59:25,990 --> 00:59:24,480
missions of their own so they can't

1636
00:59:28,069 --> 00:59:26,000
spend all their time just relaying

1637
00:59:30,150 --> 00:59:28,079
telemetry for us uh there have been

1638
00:59:32,069 --> 00:59:30,160
proposals of for instance a mars telecom

1639
00:59:34,470 --> 00:59:32,079
orbiter that was you know a few years

1640
00:59:37,510 --> 00:59:34,480
back to put an asset around mars that

1641
00:59:39,349 --> 00:59:37,520
would provide dedicated telemetry

1642
00:59:40,870 --> 00:59:39,359
i don't think we have to have that to

1643
00:59:43,510 --> 00:59:40,880
start making progress in this direction

1644
00:59:45,349 --> 00:59:43,520
but it would require

1645
00:59:46,549 --> 00:59:45,359
some pretty creative thinking about how

1646
00:59:49,270 --> 00:59:46,559
to increase the frequency of

1647
00:59:50,309 --> 00:59:49,280
communication with the robots another

1648
00:59:53,349 --> 00:59:50,319
challenge by the way is that

1649
00:59:55,270 --> 00:59:53,359
communication is expensive

1650
00:59:56,470 --> 00:59:55,280
in terms of power by the robot on the

1651
00:59:58,150 --> 00:59:56,480
surface so

1652
01:00:00,710 --> 00:59:58,160
we we also need to think about power

1653
01:00:03,190 --> 01:00:00,720
systems that can support more frequent

1654
01:00:10,150 --> 01:00:03,200
communication by robot on mars

1655
01:00:12,630 --> 01:00:11,190
oh

1656
01:00:16,710 --> 01:00:12,640
mark's bringing here a couple questions

1657
01:00:23,030 --> 01:00:19,430
so first one says can we use this

1658
01:00:25,190 --> 01:00:23,040
technology to fly an airplane on marsh

1659
01:00:26,230 --> 01:00:25,200
and i would say absolutely

1660
01:00:29,829 --> 01:00:26,240
actually the first thing that makes me

1661
01:00:33,030 --> 01:00:29,839
think of is

1662
01:00:34,549 --> 01:00:33,040
a titan airship actually one of my

1663
01:00:36,950 --> 01:00:34,559
favorite ideas that has been floated

1664
01:00:39,510 --> 01:00:36,960
around jpl in the past but focusing on

1665
01:00:42,470 --> 01:00:39,520
mars for for a second uh

1666
01:00:44,470 --> 01:00:42,480
you're not going to control you know a

1667
01:00:46,390 --> 01:00:44,480
dog fighter right you know

1668
01:00:47,829 --> 01:00:46,400
banking and turning and looping and you

1669
01:00:49,190 --> 01:00:47,839
know that's obviously not something you

1670
01:00:52,230 --> 01:00:49,200
want to do with five to 20 minutes of

1671
01:00:54,150 --> 01:00:52,240
time delay however if your airplane

1672
01:00:55,589 --> 01:00:54,160
is really a robot which it would

1673
01:00:56,630 --> 01:00:55,599
absolutely have to be

1674
01:00:58,950 --> 01:00:56,640
then

1675
01:00:59,829 --> 01:00:58,960
what it should be accomplishing is all

1676
01:01:02,549 --> 01:00:59,839
of that

1677
01:01:04,710 --> 01:01:02,559
high frequency control itself so it's

1678
01:01:06,789 --> 01:01:04,720
reacting to the buffeting winds and it's

1679
01:01:09,750 --> 01:01:06,799
keeping itself level and what you're

1680
01:01:10,950 --> 01:01:09,760
giving it are longer range trajectories

1681
01:01:12,710 --> 01:01:10,960
for instance i want to i want to start

1682
01:01:15,270 --> 01:01:12,720
reducing my altitude now and coming

1683
01:01:16,870 --> 01:01:15,280
closer to you know this mountain or i

1684
01:01:18,789 --> 01:01:16,880
want to climb now or i want to head over

1685
01:01:20,870 --> 01:01:18,799
this way and so you can imagine you know

1686
01:01:23,109 --> 01:01:20,880
when we were plotting out those

1687
01:01:27,270 --> 01:01:23,119
paths

1688
01:01:31,510 --> 01:01:29,349
there's nothing about this approach

1689
01:01:34,150 --> 01:01:31,520
that doesn't scale to controlling

1690
01:01:36,069 --> 01:01:34,160
something in three dimensions uh

1691
01:01:37,589 --> 01:01:36,079
when you're plotting out a path on the

1692
01:01:39,670 --> 01:01:37,599
ground there you can just imagine also

1693
01:01:41,510 --> 01:01:39,680
plotting a path through the air

1694
01:01:43,829 --> 01:01:41,520
it's just that you're more supervising

1695
01:01:46,309 --> 01:01:43,839
the flight than you are actively

1696
01:01:49,829 --> 01:01:46,319
controlling and you know joysticking the

1697
01:01:54,549 --> 01:01:52,069
so the second question says is nasa

1698
01:01:56,710 --> 01:01:54,559
partnered with any gaming companies to

1699
01:01:59,349 --> 01:01:56,720
aid with exploration and the answer is

1700
01:02:01,589 --> 01:01:59,359
an emphatic yes

1701
01:02:04,710 --> 01:02:01,599
so we have a number of partnerships in

1702
01:02:06,710 --> 01:02:04,720
my lab the opslab with a variety of of

1703
01:02:09,190 --> 01:02:06,720
partners and you've seen some of the

1704
01:02:10,630 --> 01:02:09,200
examples of those partnerships today

1705
01:02:11,990 --> 01:02:10,640
there are others i'll mention just

1706
01:02:13,510 --> 01:02:12,000
briefly something that i wish i could

1707
01:02:15,670 --> 01:02:13,520
have squeezed into the talk today but

1708
01:02:16,710 --> 01:02:15,680
couldn't for time was that we actually

1709
01:02:19,430 --> 01:02:16,720
released

1710
01:02:21,829 --> 01:02:19,440
nasa's first game to a video

1711
01:02:23,589 --> 01:02:21,839
gaming console one year ago it's called

1712
01:02:26,230 --> 01:02:23,599
mars river landing and we released it

1713
01:02:29,910 --> 01:02:26,240
with microsoft to the xbox 360.

1714
01:02:31,270 --> 01:02:29,920
and with it people could land on mars

1715
01:02:33,190 --> 01:02:31,280
using

1716
01:02:34,390 --> 01:02:33,200
the connect in their their living room

1717
01:02:37,349 --> 01:02:34,400
and it was

1718
01:02:40,150 --> 01:02:37,359
actually really a phenomenal success and

1719
01:02:42,069 --> 01:02:40,160
i think those things absolutely aid with

1720
01:02:43,670 --> 01:02:42,079
exploration first of all it explains to

1721
01:02:45,510 --> 01:02:43,680
people what we're trying to accomplish

1722
01:02:47,670 --> 01:02:45,520
but i think also connects them in a way

1723
01:02:49,190 --> 01:02:47,680
that other media doesn't

1724
01:02:52,549 --> 01:02:49,200
one thing i'm really

1725
01:02:53,750 --> 01:02:52,559
fond of mentioning often is that 97

1726
01:02:57,270 --> 01:02:53,760
percent

1727
01:03:00,069 --> 01:02:57,280
of american teenagers play video games

1728
01:03:02,390 --> 01:03:00,079
97 this is the pew research center study

1729
01:03:05,349 --> 01:03:02,400
of 2008 called teens video games and

1730
01:03:06,870 --> 01:03:05,359
civics so that's in 2008 97 of them

1731
01:03:08,950 --> 01:03:06,880
played which let's just face it that's

1732
01:03:11,910 --> 01:03:08,960
just all right that's all

1733
01:03:14,230 --> 01:03:11,920
all teenagers play video games

1734
01:03:16,069 --> 01:03:14,240
how many people in this room remember

1735
01:03:17,910 --> 01:03:16,079
watching the apollo landing in their

1736
01:03:19,829 --> 01:03:17,920
living room

1737
01:03:21,910 --> 01:03:19,839
a lot of hands

1738
01:03:24,069 --> 01:03:21,920
and i know from talking to a lot of

1739
01:03:26,150 --> 01:03:24,079
people about it that a lot of people who

1740
01:03:27,670 --> 01:03:26,160
work with me

1741
01:03:28,789 --> 01:03:27,680
trace their inspiration moment the

1742
01:03:31,190 --> 01:03:28,799
moment that made them want to go work

1743
01:03:33,270 --> 01:03:31,200
for nasa to that living room to that

1744
01:03:34,950 --> 01:03:33,280
moment

1745
01:03:36,710 --> 01:03:34,960
there's something very special about

1746
01:03:39,109 --> 01:03:36,720
experiencing something like that with

1747
01:03:40,630 --> 01:03:39,119
your family in your living room and

1748
01:03:43,589 --> 01:03:40,640
video games

1749
01:03:46,630 --> 01:03:43,599
are in the living room

1750
01:03:50,069 --> 01:03:46,640
so i see it as a great way for nasa to

1751
01:03:52,150 --> 01:03:50,079
be back there and again in the future so

1752
01:03:54,150 --> 01:03:52,160
yes emphatically we are partnered with

1753
01:03:55,670 --> 01:03:54,160
video game companies and i i really

1754
01:03:58,069 --> 01:03:55,680
dearly hope that we'll see even more of

1755
01:03:59,589 --> 01:03:58,079
that in the future

1756
01:04:00,789 --> 01:03:59,599
are there any final questions i'm happy

1757
01:04:02,630 --> 01:04:00,799
that if they're

1758
01:04:04,789 --> 01:04:02,640
get more on your minds come and find me